Cross-architecture compilation on Linux

I'd love to see aarch64 Swift support on the Compiler Explorer. Right now it's limited to x86-64 because Compiler Explorer happens to use x86-64 Linux servers, and the official Swift toolchains for Linux don't support cross-architecture compilation.

I saw that SE-0387: Swift SDKs for Cross-Compilation got accepted, but I don't see any apparent implementation of it…?

I saw that @Finagolfin tried to implement this support but it was rejected by Apple…?

Is there a viable way to compile for aarch64 from x86-64 Linux today?

See also compiler-explorer issue #566: ARM compiler option for Swift.

4 Likes

They do, it's just not drop-dead simple or documented yet.

I believe it is implemented and will ship with the upcoming Swift 5.9, but can't speak to any remaining issues that might hold that back, as I have not used it yet.

That is more of an ancillary issue with placing the Swift runtime libraries for multiple architectures of one non-Darwin/Windows platform in a single Swift resource directory, which is easily worked around.

I have gone into more detail there.

1 Like

i have sort of the opposite (or the same?) problem right now where i am developing on an x86_64 host, but need to deploy applications to Graviton (aarch64) servers.

right now i am working around this by compiling the application in the cloud on a Graviton host, but this requires the code to be pushed to a central git repository before it can be built, which makes it difficult to iterate on the application.

i am very curious if there is a simple way to cross-compile on Linux today.

Linux or macOS?

If you are iterating with small source changes and then want to run the tests on a different platform architecture, I would think pushing small source diffs to that central git would be faster than pushing a cross-compiled test binary. If you simply want to make sure the source still builds okay for the other arch, then local builds may be faster.

It has been simple for awhile now, as I laid out late last year. 1. And 3. there are very easy to download from swift.org, with only 2. depending on the distro and your level of expertise with it.

Then, you simply configure all those together with a SwiftPM target destination config file, as I showed with my Android example.

Alternately, you could try the new Swift SDK generator, which currently generates linux SDK bundles from macOS, that you could then use on linux also for cross-arch compilation.

@Max_Desiatov or @ktoso, are there any plans to start distributing official linux SDK bundles on swift.org, for @taylorswift's use case of building on desktop macOS/linux and deploying to linux servers? I think this would help those using Swift on the server.

since the SDK generator is macOS only, i tried following the steps in the linked post.

i did not know what to supply for the -tools-directory, and also did not know where to obtain the glibc sysroot you mentioned. so i tried installing gcc-aarch64-linux-gnu and using the following JSON:

{ "version": 1, "target": "aarch64-unknown-linux-gnu", "toolchain-bin-dir": "/home/ubuntu/usr/bin", "sdk": "/usr/aarch64-linux-gnu", "extra-cc-flags": [ "-fPIC" ], "extra-swiftc-flags": [ "-resource-dir", "/home/ubuntu/aarch64/usr/lib/swift" ], "extra-cpp-flags": [ "-lstdc++" ] } 

where /home/ubuntu/usr contains an x86_64 swift toolchain and /home/ubuntu/aarch64/usr contains the corresponding aarch64 toolchain.

i tried to compile swift-nio-ssl, which did not work:

swift build --destination /home/ubuntu/ubuntu-22.04-aarch64.json -Xlinker -rpath -Xlinker /usr/aarch64-linux-gnu/lib/ 
In file included from /swift-nio-ssl/Sources/CNIOBoringSSL/include/CNIOBoringSSL_ssl.h:145: /swift-nio-ssl/Sources/CNIOBoringSSL/include/CNIOBoringSSL_base.h:429:10: fatal error: 'memory' file not found #include <memory> ^~~~~~~~ 1 error generated. 

i tried an apt install libstdc++-12-dev per this issue, but that did not change anything, probably because that does not install the aarch64 headers. any pointers would be appreciated…

dockerfile
FROM ubuntu:jammy ENV DEBIAN_FRONTEND noninteractive WORKDIR /home/ubuntu RUN apt update RUN apt -y dist-upgrade RUN apt update RUN apt -y install \ binutils \ git \ gnupg2 \ libc6-dev \ libcurl4-openssl-dev \ libedit2 \ libgcc-9-dev \ libsqlite3-0 \ libstdc++-9-dev \ libxml2-dev \ libz3-dev \ pkg-config \ tzdata \ unzip \ zlib1g-dev RUN apt -y install curl RUN curl https://download.swift.org/swift-5.10-release/ubuntu2204/swift-5.10-RELEASE/swift-5.10-RELEASE-ubuntu22.04.tar.gz \ -o toolchain.tar.gz RUN tar --strip-components=1 -xf toolchain.tar.gz RUN rm toolchain.tar.gz RUN ln -s $PWD/usr/bin/swift /usr/bin/swift RUN apt -y install gcc-aarch64-linux-gnu WORKDIR /home/ubuntu/aarch64 RUN curl https://download.swift.org/swift-5.10-release/ubuntu2204-aarch64/swift-5.10-RELEASE/swift-5.10-RELEASE-ubuntu22.04-aarch64.tar.gz \ -o toolchain-aarch64.tar.gz RUN tar --strip-components=1 -xf toolchain-aarch64.tar.gz RUN rm toolchain-aarch64.tar.gz WORKDIR /home/ubuntu COPY ubuntu-22.04-aarch64.json . RUN rm /home/ubuntu/aarch64/usr/lib/swift/clang RUN ln -s /home/ubuntu/usr/lib/swift/clang /home/ubuntu/aarch64/usr/lib/swift/clang 

BTW: while researching the subject, i came across Cross-compiling packages which contain macros doesn't work · Issue #6950 · apple/swift-package-manager · GitHub which suggests projects that use macros do not support cross-compilation on 5.10. while i didn’t get far enough to try on my own, that’s obviously a showstopper because the application i am trying to build uses macros.

as for compiling in the cloud, i soon found that this is only really viable for debug builds. compiling an application of sufficient complexity in release mode uses too much memory and i was not able to get anywhere close to a successful release build, at least on a t4g.small instance.

That flag is only required for Android, a way to use their patched clang when linking, so it shouldn't be needed on linux.

The glibc sysroot for AArch64 should be automatically installed with that gcc-aarch64 package, else you would not be able to use it, so it's probably one of its dependencies.

Your JSON config file looks fine, with the exception of the sdk. The Swift compiler expects that path to have two directories, <sdk>/usr/include/, containing the glibc headers, and <sdk>/usr/lib/, containing the glibc library and object files.

Ubuntu probably uses the same headers in /usr/include/ across all arches it supports, so you want the sdk to be /, then it depends where your gcc-aarch toolchain places the AArch64 libc.so.

Try cross-compiling hello world in C using that gcc-aarch64 toolchain with the -v verbose flag, then look for which libc.so it links against. If it's linking against one in /usr/lib64/aarch64-linux-gnu/ instead, add "-L", "/usr/lib64/aarch64-linux-gnu/" to your extra-swiftc-flags, just as I do for Android with a similar non-standard cross-compilation layout.

That package is a bit more complicated, as it also links against libstdc++, which most Swift packages don't. I suggest you try a package like swift-argument-parser instead, as shown in my Android doc, and only try a more complicated package like NIO SSL once that is working.

Don't forget to add the right symlink in the Swift resource directory, as mentioned in my linked post. The four -Xlinker -rpath flags should not be necessary if you are then running the resulting binary in a linux AArch64 environment.

I suggest you add the --build-tests flag to build the Swift package's test runner, then copy it over to linux AArch and run it, exactly as I document doing so for Android AArch64 in my doc.

That issue was closed this week, so the next trunk snapshot toolchain may work for this.

T4g.large instances appear to be available for $20/month: that may be your only option to build Swift for linux AArch64 now, if you don't have your own local AArch64 hardware.

2 Likes

swift-argument-parser compiles successfully when using the original /usr/aarch64-linux-gnu value for the sysroot. it does not compile when using / for the sysroot. there is some kind of problem in the headers inside the aarch64 swift resource directory.

$ cat /home/ubuntu/ubuntu-22.04-aarch64.json { "version": 1, "target": "aarch64-unknown-linux-gnu", "toolchain-bin-dir": "/home/ubuntu/usr/bin", "sdk": "/", "extra-cc-flags": [ "-fPIC" ], "extra-swiftc-flags": [ "-resource-dir", "/home/ubuntu/aarch64/usr/lib/swift" ], "extra-cpp-flags": [ "-lstdc++" ] } $ ls -l /home/ubuntu/aarch64/usr/lib/swift/clang lrwxrwxrwx 1 root root 32 Apr 19 19:52 /home/ubuntu/aarch64/usr/lib/swift/clang -> /home/ubuntu/usr/lib/swift/clang 
/home/ubuntu/aarch64/usr/lib/swift/CoreFoundation/CoreFoundation.h:25:10: error: 'sys/types.h' file not found #include <sys/types.h> ^ /home/ubuntu/aarch64/usr/lib/swift/CoreFoundation/CFStream.h:20:10: note: while building module 'CDispatch' imported from /home/ubuntu/aarch64/usr/lib/swift/CoreFoundation/CFStream.h:20: #include <dispatch/dispatch.h> ^ <module-includes>:1:10: note: in file included from <module-includes>:1: #include "dispatch.h" ^ /home/ubuntu/aarch64/usr/lib/swift/dispatch/dispatch.h:32:10: note: in file included from /home/ubuntu/aarch64/usr/lib/swift/dispatch/dispatch.h:32: #include <os/generic_unix_base.h> ^ /home/ubuntu/aarch64/usr/lib/swift/os/generic_unix_base.h:26:10: error: 'sys/param.h' file not found #include <sys/param.h> ^ <unknown>:0: error: could not build C module 'CoreFoundation' error: fatalError 

so the /usr/aarch64-linux-gnu sysroot path is probably correct.

there is not much you can do on the server that doesn’t depend on TLS, directly or indirectly. so most Swift packages we are interested in using on the server do link against libstdc++, by transitive swift-nio-ssl dependency.

i tried installing the cross-compilation headers for libstdc++.

$ apt install libstdc++-12-dev-arm64-cross 

this was not sufficient to get swift-nio-ssl compiling. i can see the header in the sysroot directory

$ ls /usr/aarch64-linux-gnu/include/c++/12/ | grep memory memory memory_resource 

so i’m not sure what’s going wrong here.

That is unusual, check if both have a param.h installed: find /usr -name param.h. What may be happening if /usr/aarch64-linux-gnu/ has its own set of headers is that the Swift ClangImporter looks in several other paths too, like <sdk>/include/ and finds the headers there instead.

You can check this by adding -v -Xcc -v to your current swift build command, which will have ClangImporter dump out all include paths it's using.

Did you build the test runner with --build-tests? If not, you haven't really figured out which sdk path works.

Why would you say that? sys/{param,types}.h come from the glibc sdk, not from the resource directory.

If the same error as before, likely something to do with how the Swift compiler only looks in certain paths relative to the sdk for the C++ headers. Use those verbose flags I listed to see what it's doing.

It sounds like the system glibc-aarch64 sysroot uses some weird non-standard layout to avoid colliding with the native x86_64 sysroot in /usr/lib/ and so on. You will have to adapt your destination config file to adjust for that, or you may be better off copying all the relevant files into a proper layout in a new /home/ubuntu/glibc-aarch64/ sysroot that you create, then passing that into the destination config as the sdk.

If you want an example of a proper C/C++ sysroot layout, you can download the Android NDK and look in its sysroot/ directory to see what the Swift compiler expects.

$ find /usr -name param.h /usr/include/linux/param.h /usr/include/x86_64-linux-gnu/asm/param.h /usr/include/x86_64-linux-gnu/sys/param.h /usr/include/x86_64-linux-gnu/bits/param.h /usr/include/asm-generic/param.h /usr/aarch64-linux-gnu/include/asm/param.h /usr/aarch64-linux-gnu/include/sys/param.h /usr/aarch64-linux-gnu/include/linux/param.h /usr/aarch64-linux-gnu/include/asm-generic/param.h /usr/aarch64-linux-gnu/include/bits/param.h 

this produces a staggering amount of output, but i’m guessing this is the part we’re interested in:

clang version 15.0.0 (https://github.com/apple/llvm-project.git 5dc9d563e5a6cd2cdd44117697dead98955ccddf) Target: aarch64-unknown-linux-gnu Thread model: posix InstalledDir: /home/ubuntu/usr/bin ignoring nonexistent directory "/usr/aarch64-linux-gnu/usr/local/include" ignoring nonexistent directory "/usr/aarch64-linux-gnu/usr/include" #include "..." search starts here: #include <...> search starts here: /home/ubuntu/aarch64/usr/lib/swift /home/ubuntu/aarch64/usr/lib/swift/clang/include /usr/aarch64-linux-gnu/include End of search list. 

for some reason, it is building the package plugins (GenerateManual, etc.) for x86_64 and not aarch64; not sure what’s going on there.

the /usr/aarch64-linux-gnu path still works with --build-tests.

i downloaded a copy of that, and noticed that /android-ndk-r26d/toolchains/llvm/prebuilt/linux-x86_64/sysroot/ contains a nested usr/ directory. so i tried replicating that:

mkdir /usr/aarch64-linux-gnu/usr mv /usr/aarch64-linux-gnu/bin/ /usr/aarch64-linux-gnu/usr/ mv /usr/aarch64-linux-gnu/include/ /usr/aarch64-linux-gnu/usr/ mv /usr/aarch64-linux-gnu/lib/ /usr/aarch64-linux-gnu/usr/ 

but that was not enough to compile NIOSSL successfully.

clang -cc1 version 15.0.0 based upon LLVM 15.0.0git default target x86_64-unknown-linux-gnu ignoring nonexistent directory "/usr/aarch64-linux-gnu/usr/local/include" ignoring nonexistent directory "/usr/aarch64-linux-gnu/include" #include "..." search starts here: #include <...> search starts here: /home/ubuntu/swift-nio-ssl/Sources/CNIOBoringSSL/include /home/ubuntu/usr/lib/clang/15.0.0/include /usr/aarch64-linux-gnu/usr/include End of search list. In file included from /home/ubuntu/swift-nio-ssl/Sources/CNIOBoringSSL/ssl/ssl_versions.cc:15: In file included from /home/ubuntu/swift-nio-ssl/Sources/CNIOBoringSSL/include/CNIOBoringSSL_ssl.h:145: /home/ubuntu/swift-nio-ssl/Sources/CNIOBoringSSL/include/CNIOBoringSSL_base.h:429:10: fatal error: 'memory' file not found #include <memory> ^~~~~~~~ 

however, i can observe that memory.h exists at /usr/aarch64-linux-gnu/usr/include.

# ls /usr/aarch64-linux-gnu/usr/include a.out.h assert.h endian.h finclude gnu libgen.h misc neteconet obstack.h re_comp.h sgtty.h stdlib.h tgmath.h utmp.h aio.h bits envz.h fmtmsg.h gnu-versions.h libintl.h mntent.h netinet paths.h regex.h shadow.h string.h thread_db.h utmpx.h aliases.h byteswap.h err.h fnmatch.h grp.h limits.h monetary.h netipx poll.h regexp.h signal.h strings.h threads.h values.h alloca.h complex.h errno.h fpu_control.h gshadow.h link.h mqueue.h netiucv printf.h resolv.h sound sys time.h video ar.h cpio.h error.h fstab.h iconv.h linux mtd netpacket proc_service.h rpc spawn.h syscall.h ttyent.h wait.h argp.h ctype.h execinfo.h fts.h ieee754.h locale.h net netrom protocols sched.h stab.h sysexits.h uchar.h wchar.h argz.h dirent.h fcntl.h ftw.h ifaddrs.h malloc.h netash netrose pthread.h scsi stdc-predef.h syslog.h ucontext.h wctype.h arpa dlfcn.h features-time64.h gconv.h inttypes.h math.h netatalk nfs pty.h search.h stdint.h tar.h ulimit.h wordexp.h asm drm features.h getopt.h langinfo.h mcheck.h netax25 nl_types.h pwd.h semaphore.h stdio.h termio.h unistd.h xen asm-generic elf.h fenv.h glob.h lastlog.h memory.h netdb.h nss.h rdma setjmp.h stdio_ext.h termios.h utime.h 

Yep, it shows that the last include path used is <sdk>/include/, as I suspected.

What about the test runner, built for AArch64? file .build/aarch64-unknown-linux-gnu/debug/swift-argument-parserPackageTests.xctest

That's not the header it's looking for, it wants the C++ header memory, which is at /usr/include/c++/13/memory in my linux distro. Whatever heuristic that Swift is using to find those C++ headers isn't working for you, though make sure it is there in your sysroot first.

1 Like

ah, that was exactly what i was missing. after doing some more investigation, i figured out how to cross-compile SwiftNIO SSL for Graviton. i wrote up the instructions here:

as i anticipated, i was not able to compile the actual server application because it depends on a library that uses macros, and for some reason the compiler builds the macros for the target architecture, which prevents itself from being able to run the macros to generate the library code. as it is a library i also wrote, i’m guessing my best option is to just rewrite the library to not use macros anymore, which i was already considering for the purposes of reducing compilation times.

1 Like

Thanks for writing it up, maybe it will be useful for someone else.

Cross-compilation of Swift packages that have macros has been broken since macros were enabled in the linux compiler seven months ago, but @Max_Desiatov pushed a fix to 6.1 trunk this week. Hopefully, that gets backported to 6.0 or a 5.10 patch release if it works well.

In the meantime, that is why I suggested your best bet may be to rent out AArch64 hardware, like that t4g.large instance.

2 Likes

a cherry-picked 5.10.1 would be really great. who is in charge of that?

First we have to make sure it's working, as that fix still has other build errors when I cross-compile for Android. Download a trunk snapshot toolchain and see if it works for your cross-arch usecase now.

Hi @Finagolfin I've landed here after trying to pick up work from `swift-runtime: unable to suspend thread` when compiling in QEMU and use a cross-compile approach rather than QEMU. Thanks @taylorswift for your blog post which encouraged me to try this again!

As a reminder, I'm trying to build GitHub - grpc/grpc-swift: The Swift language implementation of gRPC. for aarch64 on a x86_64 Linux host in Docker. I have prepared the following JSON document named ubuntu-24.04-aarch64.json with the build config:

{ "version": 1, "target": "aarch64-unknown-linux-gnu", "toolchain-bin-dir": "/usr/bin", "sdk": "/usr/aarch64-linux-gnu", "extra-cc-flags": [ "-fPIC" ], "extra-cpp-flags": [ "-lstdc++", "-I", "/usr/aarch64-linux-gnu/include/c++/13", "-I", "/usr/aarch64-linux-gnu/include/c++/13/aarch64-linux-gnu/" ], "extra-swiftc-flags": [ "-resource-dir", "/root/aarch64/usr/lib/swift" ] } 

I then copy that file into the swift base image, install the aarch64 toolchain and try to cross-compile grpc-swift with a minimal Dockerfile as follows:

FROM --platform=$BUILDPLATFORM swift:6.0.3-noble AS grpc_swift RUN mkdir -p /protoc-gen-swift /grpc-swift /root/aarch64 RUN apt-get update RUN apt-get install -y \ build-essential \ curl \ gcc-aarch64-linux-gnu \ g++-aarch64-linux-gnu WORKDIR /root RUN curl -sSL https://download.swift.org/swift-6.0.3-release/ubuntu2404-aarch64/swift-6.0.3-RELEASE/swift-6.0.3-RELEASE-ubuntu24.04-aarch64.tar.gz | tar xz --strip 1 -C /root/aarch64 COPY ubuntu-24.04-aarch64.json . WORKDIR /grpc-swift RUN curl -sSL https://api.github.com/repos/grpc/grpc-swift/tarball/1.24.2 | tar xz --strip 1 -C /grpc-swift RUN swift build -c release --product protoc-gen-swift --destination /root/ubuntu-24.04-aarch64.json -v 

When invoked with docker buildx build . this fails with an Scrt1.o: unsupported ELF machine number 183 error:

65.82 error: link command failed with exit code 1 (use -v to see invocation) 65.82 clang version 17.0.0 (https://github.com/swiftlang/llvm-project.git 2e6139970eda445d9c6872c0ca293088b4e63dd2) 65.82 Target: aarch64-unknown-linux-gnu 65.82 Thread model: posix 65.82 InstalledDir: /usr/bin 65.82 Found candidate GCC installation: /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13 65.82 Selected GCC installation: /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13 65.82 Candidate multilib: .;@m64 65.82 Selected multilib: .;@m64 65.82 "/usr/bin/ld.gold" -pie -EL --hash-style=gnu --eh-frame-hdr -m aarch64linux -dynamic-linker /lib/ld-linux-aarch64.so.1 -o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc-gen-swift /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/lib/Scrt1.o /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/lib/crti.o /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/crtbeginS.o -L/root/aarch64/usr/lib/swift/linux -L/grpc-swift/.build/aarch64-unknown-linux-gnu/release -L/usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13 -L/usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../lib64 -L/lib/aarch64-linux-gnu -L/lib/../lib64 -L/usr/lib/aarch64-linux-gnu -L/usr/lib/../lib64 -L/usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/lib -L/lib -L/usr/lib -rpath /root/aarch64/usr/lib/swift/linux /root/aarch64/usr/lib/swift/linux/aarch64/swiftrt.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/AnyMessageStorage.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/AnyUnpackError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/AsyncMessageSequence.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryDecoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryDecodingError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryDecodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryDelimited.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryEncoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryEncodingError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryEncodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryEncodingSizeVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/BinaryEncodingVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/CustomJSONCodable.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Decoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/DoubleParser.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Enum.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ExtensibleMessage.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ExtensionFieldValueSet.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ExtensionFields.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ExtensionMap.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/FieldTag.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/FieldTypes.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Any+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Any+Registry.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Duration+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_FieldMask+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_ListValue+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_NullValue+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Struct+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Timestamp+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Value+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Google_Protobuf_Wrappers+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/HashVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Internal.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONDecoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONDecodingError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONDecodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONEncoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONEncodingError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONEncodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONEncodingVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONMapEncodingVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/JSONScanner.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/MathUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+AnyAdditions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+BinaryAdditions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+BinaryAdditions_Data.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+FieldMask.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+JSONAdditions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+JSONAdditions_Data.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+JSONArrayAdditions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+JSONArrayAdditions_Data.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message+TextFormatAdditions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Message.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/MessageExtension.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/NameMap.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/PathDecoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/PathVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ProtoNameProviding.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ProtobufAPIVersionCheck.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ProtobufMap.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/SelectiveVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/SimpleExtensionMap.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/StringUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/SwiftProtobufContiguousBytes.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/SwiftProtobufError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatDecoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatDecodingError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatDecodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatEncoder.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatEncodingOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatEncodingVisitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TextFormatScanner.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/TimeUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/UnknownStorage.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/UnsafeRawPointer+Shims.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Varint.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Version.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/Visitor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/WireFormat.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/ZigZag.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/any.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/api.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/descriptor.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/duration.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/empty.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/field_mask.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/resource_bundle_accessor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/source_context.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/struct.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/timestamp.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/type.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobuf.build/wrappers.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/CodeGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/CodeGeneratorParameter.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/CodePrinter.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/Descriptor+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/Descriptor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/FeatureResolver.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/FieldNumbers.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/GeneratorOutputs.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/Google_Protobuf_Compiler_CodeGeneratorResponse+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/Google_Protobuf_Edition+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/Google_Protobuf_SourceCodeInfo+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/NamingUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/PluginLibEditionDefaults.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/ProtoCompilerContext.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/ProtoFileToModuleMappings.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/ProvidesDeprecationComment.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/ProvidesLocationPath.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/ProvidesSourceCodeLocation.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/StandardErrorOutputStream.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/StringUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/SwiftLanguage.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/SwiftProtobufInfo.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/SwiftProtobufNamer.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/UnicodeScalar+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/plugin.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/resource_bundle_accessor.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/SwiftProtobufPluginLibrary.build/swift_protobuf_module_mappings.pb.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/CommandLine+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/Descriptor+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/EnumGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/ExtensionSetGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/FieldGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/FileGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/FileIo.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/GenerationError.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/GeneratorOptions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/Google_Protobuf_FileDescriptorProto+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/MessageFieldGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/MessageGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/MessageStorageClassGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/MessageStorageDecision.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/OneofGenerator.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/ProvidesDeprecationComment+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/ProvidesSourceCodeLocation+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/Range+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/StringUtils.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/SwiftGeneratorPlugin.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/SwiftProtobufNamer+Extensions.swift.o /grpc-swift/.build/aarch64-unknown-linux-gnu/release/protoc_gen_swift.build/Version.swift.o -lswiftCore -lswift_Concurrency -lswift_StringProcessing -lswift_RegexParser -lswiftGlibc -lBlocksRuntime -ldispatch -lswiftDispatch -lFoundation -lFoundationEssentials -lFoundationInternationalization -lm -lpthread -lutil -ldl -lswiftCore --gc-sections --defsym main=protoc_gen_swift_main "-rpath=\$ORIGIN" -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/crtendS.o /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/lib/crtn.o 65.82 /usr/bin/ld.gold: fatal error: /usr/bin/../lib/gcc-cross/aarch64-linux-gnu/13/../../../../aarch64-linux-gnu/lib/Scrt1.o: unsupported ELF machine number 183 65.82 clang: error: linker command failed with exit code 1 (use -v to see invocation) 

I feel like I'm really close here, but I'm not sure why the linker is failing. I previously had a lot of hacky linker stuff going on like this, but I'm not sure if any of it is still necessary with Swift 6. Can anyone help me get this over the line?

I was able to work around this using the Swift 6 SDK as follows:

FROM --platform=$BUILDPLATFORM swift:${SWIFT_IMAGE_VERSION}-noble AS grpc_swift ARG GRPC_SWIFT_VERSION SWIFT_IMAGE_VERSION RUN apt-get update && \ apt-get install -y curl RUN swift sdk install \ https://download.swift.org/swift-${SWIFT_IMAGE_VERSION}-release/static-sdk/swift-${SWIFT_IMAGE_VERSION}-RELEASE/swift-${SWIFT_IMAGE_VERSION}-RELEASE_static-linux-0.0.1.artifactbundle.tar.gz \ --checksum 67f765e0030e661a7450f7e4877cfe008db4f57f177d5a08a6e26fd661cdd0bd RUN mkdir -p /grpc-swift RUN curl -sSL https://api.github.com/repos/grpc/grpc-swift/tarball/${GRPC_SWIFT_VERSION} | tar xz --strip 1 -C /grpc-swift WORKDIR /grpc-swift ARG TARGETOS TARGETARCH RUN <<EOF case ${TARGETARCH} in "amd64") SWIFTARCH=x86_64 ;; "arm64") SWIFTARCH=aarch64 ;; *) echo "ERROR: Machine arch ${TARGETARCH} not supported." ;; esac swift build -c release --product protoc-gen-swift --swift-sdk $SWIFTARCH-swift-linux-musl swift build -c release --product protoc-gen-grpc-swift --swift-sdk $SWIFTARCH-swift-linux-musl install -D /grpc-swift/.build/release/protoc-gen-swift /out/usr/bin/protoc-gen-swift && \ install -D /grpc-swift/.build/release/protoc-gen-grpc-swift /out/usr/bin/protoc-gen-grpc-swift EOF 

This results in two very large binaries including libraries and the Swift runtime however, it would be better to be able to dynamically link these. Is this possible using the SDK?

I’m not sure about dynamic linking, but you can at least strip the binaries using the strip command if you have build-essential packages installed. This strips debug symbols from the binaries and unused libraries as well too, I believe.

I am also interested in dynamic linking but as we’ve found, it’s somewhat incomplete in swiftpm right now.

Your sdk looks off: the Swift toolchain expects a directory containing <sdk>/usr/include/, <sdk>/usr/lib/, and so on, which I doubt /usr/aarch64-linux-gnu/ contains. I'd simply try / as the sdk first and then look for workarounds if that doesn't work.

No, the static Musl SDK doesn't contain dynamic libraries, so you'd have to go back to your original approach above.

In what way? Even after the release of the static Musl SDK, dynamic linking is much better supported by SwiftPM, for example, it doesn't even support linking the static runtime libraries into dynamic library products.

1 Like

Hi @Finagolfin I'm having another try at cross-compiling a dynamically linked binary for grpc-swift under Ubuntu Linux 24.04, and then running patchelf to patch the binaries to run under Alpine Linux. My JSON file currently looks like this:

{ "version": 1, "target": "aarch64-unknown-linux-gnu", "toolchain-bin-dir": "/usr/aarch64-linux-gnu/bin", "sdk": "/root/aarch64/usr/lib/swift", "extra-cc-flags": [ "-fPIC" ], "extra-cpp-flags": [ "-lstdc++", "-I", "/usr/aarch64-linux-gnu/include/c++/13", "-I", "/usr/aarch64-linux-gnu/include/c++/13/aarch64-linux-gnu/" ], "extra-swiftc-flags": [ "-resource-dir", "/root/aarch64/usr/lib/swift" ] } 

Build succeeds with no errors. When I try to run the resulting binary, I get this error:

protoc-gen-swift: error while loading shared libraries: libswiftCore.so: cannot open shared object file: No such file or directory 

If I add a step to my build Dockerfile to run ldd on the binary to see what patchelf is trying to patch in, I get the following result showing libswiftCore.so is indeed not found:

#23 [grpc_swift 5/5] RUN ldd /protoc-gen-swift/protoc-gen-swift #23 0.389 linux-vdso.so.1 (0x00007f2d30710000) #23 0.390 libswiftCore.so => not found #23 0.390 libswift_Concurrency.so => not found #23 0.390 libswift_StringProcessing.so => not found #23 0.390 libswift_RegexParser.so => not found #23 0.390 libswiftGlibc.so => not found #23 0.390 libBlocksRuntime.so => not found #23 0.390 libdispatch.so => not found #23 0.390 libswiftDispatch.so => not found #23 0.390 libFoundation.so => not found #23 0.390 libFoundationEssentials.so => not found #23 0.390 libFoundationInternationalization.so => not found #23 0.390 libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x00007f2d27af0000) #23 0.390 libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x00007f2d27930000) #23 0.390 /lib/ld-linux-aarch64.so.1 (0x00007f2d306b0000) #23 DONE 0.5s 

Why doesn't the compiler error instead of building a binary that links to non-existent files? Stepping inside the build container, I have unpacked the swift-6.1-RELEASE-ubuntu24.04-aarch64.tar.gz to /root/aarch64. I can see all of the "not found" .so files located in /root/aarch64/usr/lib/swift/linux. No matter how I mess around with the toolchain-bin-dir, sdk and resource-dir variables, I cannot build a binary that correctly links to these files. In particular, I have tried setting /, /root/aarch64 and every intermediate directory up to /root/aarch64/usr/lib/swift/linux.

Ubuntu installs the aarch64 toolchain under /usr/aarch64-linux-gnu, and I believe I'm pointing to it correctly. Any idea what I might be doing wrong? My full Dockerfile is below.

FROM --platform=$BUILDPLATFORM swift:${SWIFT_IMAGE_VERSION}-noble AS grpc_swift_builder ARG PROTOC_GEN_SWIFT_VERSION ARG SWIFT_IMAGE_VERSION ARG SWIFT_SDK_CHECKSUM RUN apt-get update && \ apt-get install -y build-essential curl file RUN mkdir -p /grpc-swift RUN curl -sSL https://api.github.com/repos/grpc/grpc-swift/tarball/${PROTOC_GEN_SWIFT_VERSION} | tar xz --strip 1 -C /grpc-swift WORKDIR /grpc-swift ARG TARGETOS TARGETARCH RUN <<EOF if [ "${TARGETARCH}" = "arm64" ]; then SWIFT_VERSION=$(echo ${SWIFT_IMAGE_VERSION} | sed -E 's/([0-9]+\.[0-9]+)\.0/\1/') mkdir -p /root/aarch64 curl -sSL https://download.swift.org/swift-$SWIFT_VERSION-release/ubuntu2404-aarch64/swift-$SWIFT_VERSION-RELEASE/swift-$SWIFT_VERSION-RELEASE-ubuntu24.04-aarch64.tar.gz | tar xz --strip 1 -C /root/aarch64 apt-get install -y \ gcc-aarch64-linux-gnu \ g++-aarch64-linux-gnu ln -sf /usr/bin/aarch64-linux-gnu-ld.gold /usr/bin/ld.gold fi EOF COPY ubuntu-24.04-aarch64.json /root/ubuntu-24.04-aarch64.json RUN <<EOF if [ "${TARGETARCH}" = "arm64" ]; then SWIFTBUILD="swift build --destination /root/ubuntu-24.04-aarch64.json" else SWIFTBUILD="swift build" fi $SWIFTBUILD -c release --product protoc-gen-swift EOF RUN mkdir -p /protoc-gen-swift RUN install -D /grpc-swift/.build/release/protoc-gen-swift /protoc-gen-swift/protoc-gen-swift FROM swift:${SWIFT_IMAGE_VERSION}-noble AS grpc_swift ARG TARGETARCH RUN apt-get update && \ apt-get install -y patchelf file COPY --from=grpc_swift_builder /protoc-gen-swift/ /protoc-gen-swift/ # Debug to check if linking worked before we try to run patchelf RUN ldd /protoc-gen-swift/protoc-gen-swift RUN <<EOF case ${TARGETARCH} in "amd64") SWIFT_LIB_DIR=/lib64 && SWIFT_LINKER=ld-linux-x86-64.so.2 ;; "arm64") SWIFT_LIB_DIR=/lib && SWIFT_LINKER=ld-linux-aarch64.so.1 ;; *) echo "ERROR: Machine arch ${TARGETARCH} not supported." ;; esac cp ${SWIFT_LIB_DIR}/${SWIFT_LINKER} \ $(ldd /protoc-gen-swift/protoc-gen-swift | awk '{print $3}' | grep /lib | sort | uniq) /protoc-gen-swift/ find /protoc-gen-swift/ -name 'lib*.so*' -exec patchelf --set-rpath /protoc-gen-swift {} \; for p in protoc-gen-swift; do patchelf --set-interpreter /protoc-gen-swift/${SWIFT_LINKER} /protoc-gen-swift/${p} done EOF 

I'm surprised it built at all, as that "sdk" path is wrong.

Sure, easy one, the rpath is wrong because of this line you added: find /protoc-gen-swift/ -name 'lib*.so*' -exec patchelf --set-rpath /protoc-gen-swift {}.

There are two paths that get passed in to the linker where it should look for Swift runtime libraries, -L /some/path for when building, then -rpath /some/rpath for the binary to look in when running later. Both are generated automatically by the Swift compiler based on the -resource-dir you passed in, you can check what they are by passing -v when building.

The problem is that you're erasing that auto-generated rpath when inserting /protoc-gen-swift, so make sure it stays by changing that to --set-rpath /protoc-gen-swift:/root/aarch64/usr/lib/swift/linux instead.