Revert "Merge pull request #40595 from hamishknight/straw-bales"

[aschwaighofer]

This reverts commit a67a043, reversing
changes made to 9965df7.

This commit or the earlier commit this commit is based on (#40531) broke the
incremental bot.

[aschwaighofer]

@swift-ci Please test

[aschwaighofer]

preset=buildbot_incremental,tools=RA,stdlib=RA
@swift-ci Please test with preset macOS

[aschwaighofer]

incremental builder: https://ci.swift.org/job/oss-swift-incremental-RA-macos/14501/

[aschwaighofer]

20:58:28 -- Configuring done 20:58:28 CMake Error at stdlib/cmake/modules/AddSwiftStdlib.cmake:923 (add_library): 20:58:31 No SOURCES given to target: swift_MatchingEngine-watchos-armv7k 20:58:31 Call Stack (most recent call first): 20:58:31 stdlib/cmake/modules/AddSwiftStdlib.cmake:2045 (add_swift_target_library_single) 20:58:31 stdlib/public/MatchingEngine/CMakeLists.txt:25 (add_swift_target_library) 

[milseman]

Reverting this is likely to cause significantly more breakage overall, so let’s see if we can’t fix the CI. Is it just not getting the sources, or is there a compilation error?

[milseman]

The incremental builder needs to run update checkout, that is, it needs a fullish build once to get the new sources

[rxwei]

cc @drewmaxwell-apple

[aschwaighofer]

I don't know why the failure is. But nearly all bots are red right now.

[aschwaighofer]

There seem to be different issues. I have seen the issue mentioned above and here is a different one:

https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158/console

10:56:23 <unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found 10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:4:40: error: cannot find 'libswiftParseRegexLiteral' in scope 10:56:23 Parser_registerRegexLiteralParsingFn(libswiftParseRegexLiteral) 10:56:23 ^~~~~~~~~~~~~~~~~~~~~~~~~ 10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:5:39: error: cannot find 'libswiftLexRegexLiteral' in scope 10:56:23 Parser_registerRegexLiteralLexingFn(libswiftLexRegexLiteral) 10:56:23 ^~~~~~~~~~~~~~~~~~~~~~~ 

[aschwaighofer]

Here is a -- probably incomplete -- list of bots currently failing with either of these two errors:

https://ci.swift.org/job/oss-swift-incremental-RA-macos/14503/ https://ci.swift.org/job/oss-swift-incremental-RA-macos-apple-silicon/1969 https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158 https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04-long-test/9401/ https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-18_04/13485 https://ci.swift.org/job/oss-swift-pr-test/19434 https://ci.swift.org/job/oss-swift-incremental-ASAN-RA-macos/6148 https://ci.swift.org/job/oss-swift_tools-RA_stdlib-RDA_long-test/5536 https://ci.swift.org/job/oss-swift_tools-RA_stdlib-RDA_test-simulator/4172 https://ci.swift.org/job/oss-swift_tools-R_stdlib-RD_test-simulator/6761 https://ci.swift.org/job/oss-swift-test-stdlib-with-toolchain/746 https://ci.swift.org/job/swift-main-source-compat-suite/6355 https://ci.swift.org/view/Dashboard/job/swift-main-source-compat-suite-debug/4393 

[rxwei]

There seem to be different issues. I have seen the issue mentioned above and here is a different one:

https://ci.swift.org/job/oss-swift-incremental-RA-linux-ubuntu-16_04/18158/console

10:56:23 <unknown>:0: warning: unable to perform implicit import of "_Concurrency" module: no such module found 10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:4:40: error: cannot find 'libswiftParseRegexLiteral' in scope 10:56:23 Parser_registerRegexLiteralParsingFn(libswiftParseRegexLiteral) 10:56:23 ^~~~~~~~~~~~~~~~~~~~~~~~~ 10:56:23 /home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift/libswift/Sources/ExperimentalRegex/Regex.swift:5:39: error: cannot find 'libswiftLexRegexLiteral' in scope 10:56:23 Parser_registerRegexLiteralLexingFn(libswiftLexRegexLiteral) 10:56:23 ^~~~~~~~~~~~~~~~~~~~~~~ 

This one has found the sources and has SWIFT_ENABLE_EXPERIMENTAL_STRING_PROCESSING=TRUE, but did not try to build either _MatchingEngine or _StringProcessing.

10:47:37 -- Using Experimental String Processing library for _MatchingEngine (/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift-experimental-string-processing). 10:47:37 -- Using Experimental String Processing library for _StringProcessing (/home/buildnode/jenkins/workspace/oss-swift-incremental-RA-linux-ubuntu-16_04/swift-experimental-string-processing).

I wonder if it just needs a clean build.

[aschwaighofer]

The build bot should have been cleaned in between 2 days. That at least would be my understanding of doing things sensibly (i.e the bots clean the workspace overnight).

The issue did not seem to resolve itself since yesterday, though. I can't tell why.

That said I tried a clean build locally with update-checkout --clone-with-ssh --scheme main and this build did not reproduce the issue and was fine.
Neither would it appear we had issue when testing the PR, right?

It could be that the bots don't use update-checkout and the bot needs to be configured to checkout whatever repo we are missing (but then how could your PR have passed?). Or we indeed just need a clean workspace build and that has not happened overnight for whatever reason.

I will wait until tomorrow to see what happens tonight.

[aschwaighofer]

According to the following run the bot macOS-31 was cleaned today the last time: https://ci.swift.org/job/clean-workspace-macOS-bk/102923/console

That also seems to be the bot that the incremental macOS bot runs on: https://ci.swift.org/job/oss-swift-incremental-RA-macos/14503/.

I am going to try to trigger a build now by committing an NFC change. This build should be on a clean workspace if it runs on macOS-31.

[aschwaighofer]

Okay, this run https://ci.swift.org/job/oss-swift-incremental-RA-macos/14504 is indeed running on macOS-31 now. So it should be running a clean build because of https://ci.swift.org/job/clean-workspace-macOS-bk/102923.

[rxwei]

Mishal informed us that new repos need to be manually added to the CI and that @drewmaxwell-apple should be able to help add it. Drew, could you help us add it when you get a chance?

[aschwaighofer]

Okay. Sounds like we have found a way to not break the bots: first we configure the various CI configurations to check out the necessary repo. Once this is complete we can re-commit this change.