Skip to content

feat: use debian base image for java-library-generation #3817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 30, 2025
+5 −83
Merged

feat: use debian base image for java-library-generation #3817

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7d54841
feat: use debian base image for `java-library-generation`
diegomarquezp May 29, 2025
45b165d
feat: update Airlock image with debian base image
diegomarquezp May 29, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 2 additions & 41 deletions .cloudbuild/library_generation/library_generation.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,27 +34,7 @@ RUN mvn install -B -ntp -DskipTests -Dclirr.skip -Dcheckstyle.skip
RUN cp "/root/.m2/repository/com/google/api/gapic-generator-java/${DOCKER_GAPIC_GENERATOR_VERSION}/gapic-generator-java-${DOCKER_GAPIC_GENERATOR_VERSION}.jar" \
"./gapic-generator-java.jar"

FROM docker.io/library/alpine:3.21.2@sha256:56fa17d2a7e7f168a043a2712e63aed1f8543aeafdcee47c58dcffe38ed51099 as glibc-compat
Copy link
Contributor

@blakeli0 blakeli0 May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remind me why we chose alpine in the first place?
Copy link
Contributor Author

@diegomarquezp diegomarquezp May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alpine is promoted as a lightweight base image that happens to be secure due to it's smaller attack surface.

Moving to debian-slim gives us a slightly heavier image but compatible with glibc.

RUN apk update && apk add git sudo
# This SHA is the latest known-to-work version of this binary compatibility tool
ARG GLIB_MUS_SHA=e94aca542e3ab08b42aa0b0d6e72478b935bb8e8
WORKDIR /home

# Install compatibility layer to run glibc-based programs (such as the
# grpc plugin).
# Alpine, by default, only supports musl-based binaries, and there is no public
# downloadable distribution of the grpc plugin that is Alpine (musl) compatible.
# This is one of the recommended approaches to ensure glibc-compatibility
# as per https://wiki.alpinelinux.org/wiki/Running_glibc_programs
RUN git clone https://gitlab.com/manoel-linux1/GlibMus-HQ.git
WORKDIR /home/GlibMus-HQ
# We lock the tool to the latest known-to-work version
RUN git checkout "${GLIB_MUS_SHA}"
RUN chmod a+x compile-x86_64-alpine-linux.sh
RUN sh compile-x86_64-alpine-linux.sh

FROM docker.io/library/python:3.13.2-alpine3.20@sha256:816feb29731cdee64b15b0ae91dd9f1cbc36765984ff8ea85a3d90f064417237 as final
FROM docker.io/library/python:3.13.2-slim@sha256:6b3223eb4d93718828223966ad316909c39813dee3ee9395204940500792b740 as final

ARG OWLBOT_CLI_COMMITTISH=3a68a9c0de318784b3aefadcc502a6521b3f1bc5
ARG PROTOC_VERSION=25.5
Expand All @@ -63,28 +43,10 @@ ENV HOME=/home
ENV OS_ARCHITECTURE="linux-x86_64"

# install OS tools
RUN apk update && apk add unzip curl rsync openjdk17 jq bash nodejs npm git
RUN apt update && apt install -y curl unzip rsync jq nodejs npm git openjdk-17-jdk

SHELL [ "/bin/bash", "-c" ]

# Copy glibc shared objects to enable execution of the grpc plugin.
# This list was obtained via `libtree -pvvv /grpc/*` in the final container as
# well as inspecting the modifications done by compile-x86_64-alpine-linux.sh
# in the glibc-compat stage using the `dive` command.
COPY --from=glibc-compat /etc/libgcc* /etc/
COPY --from=glibc-compat /lib64/ld-linux-x86-64.so.2 /lib64/
COPY --from=glibc-compat /lib/GLIBCFAKE.so.0 /lib/
COPY --from=glibc-compat /lib/ld-linux-x86-64.so.2 /lib/
COPY --from=glibc-compat /lib/libpthread* /lib/
COPY --from=glibc-compat /lib/libucontext* /lib/
COPY --from=glibc-compat /lib/libc.* /lib/
COPY --from=glibc-compat /usr/lib/libgcc* /usr/lib/
COPY --from=glibc-compat /usr/lib/libstdc* /usr/lib/
COPY --from=glibc-compat /usr/lib/libobstack* /usr/lib/
COPY --from=glibc-compat /lib/libm.so.6 /usr/lib/
COPY --from=glibc-compat /usr/lib/libucontext.so.1 /usr/lib/


# copy source code
COPY hermetic_build/common /src/common
COPY hermetic_build/library_generation /src/library_generation
Expand Down Expand Up @@ -121,7 +83,6 @@ RUN git checkout "${OWLBOT_CLI_COMMITTISH}"
RUN npm i && npm run compile && npm link
RUN owl-bot copy-code --version
RUN chmod o+rx $(which owl-bot)
RUN apk del -r npm && apk cache clean

# copy the Java formatter
COPY --from=ggj-build "/google-java-format.jar" "${HOME}"/.library_generation/google-java-format.jar
Expand Down
45 changes: 3 additions & 42 deletions .cloudbuild/library_generation/library_generation_airlock.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,29 +34,8 @@ RUN mvn install -B -ntp -DskipTests -Dclirr.skip -Dcheckstyle.skip -Dfmt.skip
RUN cp "/root/.m2/repository/com/google/api/gapic-generator-java/${DOCKER_GAPIC_GENERATOR_VERSION}/gapic-generator-java-${DOCKER_GAPIC_GENERATOR_VERSION}.jar" \
"./gapic-generator-java.jar"

# 3.20.1
FROM us-docker.pkg.dev/artifact-foundry-prod/docker-3p-trusted/alpine@sha256:dabf91b69c191a1a0a1628fd6bdd029c0c4018041c7f052870bb13c5a222ae76 as glibc-compat

RUN apk update && apk add git sudo
# This SHA is the latest known-to-work version of this binary compatibility tool
ARG GLIB_MUS_SHA=e94aca542e3ab08b42aa0b0d6e72478b935bb8e8
WORKDIR /home

# Install compatibility layer to run glibc-based programs (such as the
# grpc plugin).
# Alpine, by default, only supports musl-based binaries, and there is no public
# downloadable distribution of the grpc plugin that is Alpine (musl) compatible.
# This is one of the recommended approaches to ensure glibc-compatibility
# as per https://wiki.alpinelinux.org/wiki/Running_glibc_programs
RUN git clone https://gitlab.com/manoel-linux1/GlibMus-HQ.git
WORKDIR /home/GlibMus-HQ
# We lock the tool to the latest known-to-work version
RUN git checkout "${GLIB_MUS_SHA}"
RUN chmod a+x compile-x86_64-alpine-linux.sh
RUN sh compile-x86_64-alpine-linux.sh

# 3.12.7-alpine3.20
FROM us-docker.pkg.dev/artifact-foundry-prod/docker-3p-trusted/python@sha256:b83d5ec7274bee17d2f4bd0bfbb082f156241e4513f0a37c70500e1763b1d90d as final
# 3.12.3-slim-bookworm
Copy link
Contributor

@zhumin8 zhumin8 May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this older version of the public image above? docker.io/library/python:3.13.2-slim
Copy link
Contributor Author

@diegomarquezp diegomarquezp May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It unfortunately is, since Airlock has a delay. I thought it's a good temporary base image in the meantime.
I'm planning to update to the slim image I requested to import into Airlock.
FROM us-docker.pkg.dev/artifact-foundry-prod/docker-3p-trusted/python@sha256:afc139a0a640942491ec481ad8dda10f2c5b753f5c969393b12480155fe15a63 as final

ARG OWLBOT_CLI_COMMITTISH=3a68a9c0de318784b3aefadcc502a6521b3f1bc5
ARG PROTOC_VERSION=25.5
Expand All @@ -65,27 +44,10 @@ ENV HOME=/home
ENV OS_ARCHITECTURE="linux-x86_64"

# install OS tools
RUN apk update && apk add unzip curl rsync openjdk17 jq bash nodejs npm git
RUN apt update && apt install -y curl unzip rsync jq nodejs npm git openjdk-17-jdk

SHELL [ "/bin/bash", "-c" ]

# Copy glibc shared objects to enable execution of the grpc plugin.
# This list was obtained via `libtree -pvvv /grpc/*` in the final container as
# well as inspecting the modifications done by compile-x86_64-alpine-linux.sh
# in the glibc-compat stage using the `dive` command.
COPY --from=glibc-compat /etc/libgcc* /etc/
COPY --from=glibc-compat /lib64/ld-linux-x86-64.so.2 /lib64/
COPY --from=glibc-compat /lib/GLIBCFAKE.so.0 /lib/
COPY --from=glibc-compat /lib/ld-linux-x86-64.so.2 /lib/
COPY --from=glibc-compat /lib/libpthread* /lib/
COPY --from=glibc-compat /lib/libucontext* /lib/
COPY --from=glibc-compat /lib/libc.* /lib/
COPY --from=glibc-compat /usr/lib/libgcc* /usr/lib/
COPY --from=glibc-compat /usr/lib/libstdc* /usr/lib/
COPY --from=glibc-compat /usr/lib/libobstack* /usr/lib/
COPY --from=glibc-compat /lib/libm.so.6 /usr/lib/


# copy source code
COPY hermetic_build/common /src/common
COPY hermetic_build/library_generation /src/library_generation
Expand Down Expand Up @@ -122,7 +84,6 @@ RUN git checkout "${OWLBOT_CLI_COMMITTISH}"
RUN npm i && npm run compile && npm link
RUN owl-bot copy-code --version
RUN chmod o+rx $(which owl-bot)
RUN apk del -r npm && apk cache clean

# copy the Java formatter
COPY --from=ggj-build "/google-java-format.jar" "${HOME}"/.library_generation/google-java-format.jar
Expand Down
Loading