Skip to content

Conversation

windxu88
Copy link
Collaborator

Description

Standalone-il2cpp tests on Ubuntu 22 are unstable because Editor can freeze when building the player in this environment.
This PR:

  • removed standalone-il2cpp tests on Ubuntu from PR trigger. By doing this, we can have a more stable PR trigger
  • added a timeout value 3600 seconds to UTR command in all test jobs which should be enough. I have checked all the jobs, their run time are all under 30 minutes. After setting the timeout value, a job won't be stuck there for 6 hours.
  • added a nightly trigger which run all performance tests and standalone-il2cpp tests on Ubuntu.

Testing status & QA

Please describe the testing already done by you and what testing you request/recommend QA to execute. If you used or created any testing project please link them here too for QA.

Overall Product Risks

Please rate the potential complexity and halo effect from low to high for the reviewers. Note down potential risks to specific Editor branches if any.

  • Complexity:
  • Halo Effect:

Comments to reviewers

Please describe any additional information such as what to focus on, or historical info for the reviewers.

Checklist

Before review:

  • Changelog entry added.
    • Explains the change in Changed, Fixed, Added sections.
    • For API change contains an example snippet and/or migration example.
    • JIRA ticket linked, example (case %%). If it is a private issue, just add the case ID without a link.
    • Jira port for the next release set as "Resolved".
  • Tests added/changed, if applicable.
    • Functional tests Area_CanDoX, Area_CanDoX_EvenIfYIsTheCase, Area_WhenIDoX_AndYHappens_ThisIsTheResult.
    • Performance tests.
    • Integration tests.
  • Docs for new/changed API's.
    • Xmldoc cross references are set correctly.
    • Added explanation how the API works.
    • Usage code examples added.
    • The manual is updated, if needed.

During merge:

  • Commit message for squash-merge is prefixed with one of the list:
    • NEW: ___.
    • FIX: ___.
    • DOCS: ___.
    • CHANGE: ___.
    • RELEASE: 1.1.0-preview.3.

After merge:

  • Create forward/backward port if needed. If you are blocked from creating a forward port now please add a task to ISX-1444.
@u-pr-agent
Copy link

u-pr-agent bot commented Oct 15, 2025

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

The PR modifies CI/CD job configurations, which requires some context on the CI system, but the changes are small and well-described.
🏅 Score: 95

The PR improves CI stability by moving flaky tests to a nightly run and adds sensible timeouts, with a clean and logical implementation.
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Job Filtering

The filtering of Ubuntu jobs relies on the JobId containing the string "Ubuntu". While this works, it could be brittle if job naming conventions change in the future. It might be worth considering if a more robust method, like a tag or property on the job definition, is available.

.WithDependencies(allStandaloneIl2CppFunctionalTests.Where(d => !d.JobId.Contains("Ubuntu")) )

🤖 Helpful? Please react with 👍/👎 | Questions❓Please reach out in Slack #ask-u-pr-agent

@windxu88 windxu88 changed the title Remove standaloneil2cpp tests on Ubuntu from PR trigger and add them to nightly trigger CHANGE: Remove standaloneil2cpp tests on Ubuntu from PR trigger and add them to nightly trigger Oct 15, 2025
@codecov-github-com
Copy link

codecov-github-com bot commented Oct 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

@@ Coverage Diff @@ ## develop #2259 +/- ## =========================================== - Coverage 76.70% 76.62% -0.09%  =========================================== Files 465 472 +7 Lines 87919 88289 +370 =========================================== + Hits 67442 67653 +211  - Misses 20477 20636 +159 
Flag Coverage Δ
inputsystem_MacOS_2021.3 5.90% <ø> (-0.01%) ⬇️
inputsystem_MacOS_2022.3 5.37% <ø> (-0.01%) ⬇️
inputsystem_MacOS_2022.3_project 74.51% <ø> (-0.07%) ⬇️
inputsystem_MacOS_6000.0 5.18% <ø> (-0.01%) ⬇️
inputsystem_MacOS_6000.0_project 76.43% <ø> (-0.08%) ⬇️
inputsystem_MacOS_6000.2 5.18% <ø> (-0.01%) ⬇️
inputsystem_MacOS_6000.2_project 76.43% <ø> (-0.08%) ⬇️
inputsystem_MacOS_6000.3 5.18% <ø> (-0.01%) ⬇️
inputsystem_MacOS_6000.3_project 76.43% <ø> (-0.08%) ⬇️
inputsystem_MacOS_6000.4 5.18% <ø> (-0.01%) ⬇️
inputsystem_MacOS_6000.4_project 76.43% <ø> (-0.08%) ⬇️
inputsystem_Ubuntu_2021.3 5.91% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_2022.3 5.37% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_2022.3_project 74.32% <ø> (-0.07%) ⬇️
inputsystem_Ubuntu_6000.0 5.19% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_6000.0_project 76.24% <ø> (-0.08%) ⬇️
inputsystem_Ubuntu_6000.2 5.19% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_6000.2_project 76.24% <ø> (-0.08%) ⬇️
inputsystem_Ubuntu_6000.3 5.19% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_6000.3_project 76.24% <ø> (-0.08%) ⬇️
inputsystem_Ubuntu_6000.4 5.19% <ø> (-0.01%) ⬇️
inputsystem_Ubuntu_6000.4_project 76.24% <ø> (-0.09%) ⬇️
inputsystem_Windows_2021.3 5.90% <ø> (-0.01%) ⬇️
inputsystem_Windows_2022.3 5.37% <ø> (-0.01%) ⬇️
inputsystem_Windows_2022.3_project 74.66% <ø> (-0.07%) ⬇️
inputsystem_Windows_6000.0 5.18% <ø> (-0.01%) ⬇️
inputsystem_Windows_6000.0_project 76.57% <ø> (-0.08%) ⬇️
inputsystem_Windows_6000.2 5.18% <ø> (-0.01%) ⬇️
inputsystem_Windows_6000.2_project 76.57% <ø> (-0.08%) ⬇️
inputsystem_Windows_6000.3 5.18% <ø> (-0.01%) ⬇️
inputsystem_Windows_6000.3_project 76.57% <ø> (-0.08%) ⬇️
inputsystem_Windows_6000.4 5.18% <ø> (-0.01%) ⬇️
inputsystem_Windows_6000.4_project 76.56% <ø> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
@ekcoh ekcoh merged commit 78ed98f into develop Oct 16, 2025
110 checks passed
@ekcoh ekcoh deleted the remove_standaloneil2cpp_ubuntu_from_pr_trigger branch October 16, 2025 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants