Ruby

I have downgraded RBS to 1.6.2, but the same issue suffers (not surprisingly) debug:

Building native extensions. This could take a while... /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:95:in `run': ERROR: Failed to build gem native extension. (Gem::Ext::BuildError) current directory: /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/bin/ruby --disable\\=gems -I/builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/lib64/ruby -I /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib -r ./siteconf20211201-24843-jbuqiz.rb extconf.rb creating Makefile current directory: /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug make DESTDIR\\= clean make[1]: Entering directory '/builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug' rm -f rm -fr debug.so false *.o *.bak mkmf.log .*.time make[1]: Leaving directory '/builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug' current directory: /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug make DESTDIR\\= make[1]: Entering directory '/builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug' make[1]: *** No rule to make target '/include/x86_64-linux/ruby/config.h', needed by 'debug.o'. Stop. make[1]: Leaving directory '/builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4/ext/debug' make failed, exit code 2 Gem files will remain installed in /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/gems/debug-1.3.4 for inspection. Results logged to /builddir/build/BUILDROOT/ruby-3.1.0-0.1.20211201gita84dc9d80d.fc36.x86_64/usr/share/ruby/gems/extensions/x86_64-linux/3.1.0/debug-1.3.4/gem_make.out	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:44:in `block in make'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:36:in `each'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:36:in `make'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/ext_conf_builder.rb:63:in `block in build'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/tempfile.rb:317:in `open'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/ext_conf_builder.rb:26:in `build'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:161:in `build_extension'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:195:in `block in build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:192:in `each'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:192:in `build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/installer.rb:837:in `build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/installer.rb:318:in `install'	from ./tool/rbinstall.rb:895:in `block in install'	from ./tool/rbinstall.rb:719:in `no_write'	from ./tool/rbinstall.rb:895:in `install'	from ./tool/rbinstall.rb:1062:in `block (2 levels) in <main>'	from ./tool/rbinstall.rb:1037:in `foreach'	from ./tool/rbinstall.rb:1037:in `block in <main>'	from ./tool/rbinstall.rb:1121:in `block in <main>'	from ./tool/rbinstall.rb:1118:in `each'	from ./tool/rbinstall.rb:1118:in `<main>' /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:95:in `run': make failed, exit code 2 (Gem::InstallError)	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:44:in `block in make'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:36:in `each'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:36:in `make'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/ext_conf_builder.rb:63:in `block in build'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/tempfile.rb:317:in `open'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/ext_conf_builder.rb:26:in `build'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:161:in `build_extension'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:195:in `block in build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:192:in `each'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/ext/builder.rb:192:in `build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/installer.rb:837:in `build_extensions'	from /builddir/build/BUILD/ruby-3.1.0-a84dc9d80d/lib/rubygems/installer.rb:318:in `install'	from ./tool/rbinstall.rb:895:in `block in install'	from ./tool/rbinstall.rb:719:in `no_write'	from ./tool/rbinstall.rb:895:in `install'	from ./tool/rbinstall.rb:1062:in `block (2 levels) in <main>'	from ./tool/rbinstall.rb:1037:in `foreach'	from ./tool/rbinstall.rb:1037:in `block in <main>'	from ./tool/rbinstall.rb:1121:in `block in <main>'	from ./tool/rbinstall.rb:1118:in `each'	from ./tool/rbinstall.rb:1118:in `<main>' make: *** [uncommon.mk:383: do-install-all] Error 1 

FYI, in my environment ruby-3.1.0-preview1 can be cross-installed without error for aarch64-linux-android target (with build host being x86_64-linux-gnu).

Some observation:

$ grep 'config\.h' /var/tmp/build/arm64-v8a/lib/ruby/gems/3.1.0/gems/rbs-1.7.0.beta.5/ext/rbs_extension/Makefile ruby_headers = $(hdrdir)/ruby.h $(hdrdir)/ruby/backward.h $(hdrdir)/ruby/ruby.h $(hdrdir)/ruby/defines.h $(hdrdir)/ruby/missing.h $(hdrdir)/ruby/intern.h $(hdrdir)/ruby/st.h $(hdrdir)/ruby/subst.h $(arch_hdrdir)/ruby/config.h $ grep '^arch_hdrdir' /var/tmp/build/arm64-v8a/lib/ruby/gems/3.1.0/gems/rbs-1.7.0.beta.5/ext/rbs_extension/Makefile arch_hdrdir = /var/tmp/ruby-3.1.0-preview1/.build-android-aarch64/.ext/include/aarch64-linux-android 

So I guess in your environment arch_hdrdir in Makefile for gem extension is incorrectly defined as /include/x86_64-linux, though I don't know why.

What make command do you use?

Can you provide the full log of the build.log showing the build error as an attachment of this ticket or somewhere?

this only breaks when you run "make install DESTDIR=/some/path"

but it works nicely without. so this is a bug in how DESTDIR is passed
to the rubygems extensions build.

darix

--
Always remember:
Never accept the world as it appears to be.
Dare to see it for what it could be.
The world can always use more heroes.

I am not sure what to really blame. The place where it goes quite wrong is actually mkmf, somewhere around here:

 if not $extmk and File.exist?(($hdrdir = RbConfig::CONFIG["rubyhdrdir"]) + "/ruby/ruby.h") $topdir = $hdrdir $top_srcdir = $hdrdir $arch_hdrdir = RbConfig::CONFIG["rubyarchhdrdir"] elsif File.exist?(($hdrdir = ($top_srcdir ||= topdir) + "/include") + "/ruby.h") $topdir ||= RbConfig::CONFIG["topdir"] $arch_hdrdir = "$(extout)/include/$(arch)" else 

The first branch is used for regular cases when Ruby is already installed. The second branch is used for building extensions during build of Ruby.

Unfortunately, installing gems and building their extensions is non of these. So there should be probably created some middle path. Maybe if the mkmf could be convinced that the extension is nothing different to building extensions which are part of Ruby ...

But I think that it might make difference, if there is already installed Ruby on the system, which is not my case.

I was able to use this little hack to compile the extensions:

find .bundle -name extconf.rb -exec \ sed -i \ -e '/create_makefile/i \$arch_hdrdir = "$(hdrdir)/../.ext/include/$(arch)"' \ -e '/create_makefile/i \$DLDFLAGS << " -L#{$top_srcdir}"' {} \; 

BTW, this causes also different issues as far as I can tell:

https://github.com/ruby/ruby/pull/5206

Please answer to https://bugs.ruby-lang.org/issues/18373#note-3. I couldn't reproduce the issue by the following commands

$ cd /tmp $ git clone https://github.com/ruby/ruby.git $ cd ruby $ ./autogen.sh $ ./configure $ make $ make install DESTDIR=/tmp/local/ 

A small and self-contained procedure to reproduce this issue is needed to make progress.

mame (Yusuke Endoh) wrote in #note-12:

Please answer to https://bugs.ruby-lang.org/issues/18373#note-3. I couldn't reproduce the issue by the following commands

$ cd /tmp $ git clone https://github.com/ruby/ruby.git 

This ^^ is one issue. The starting point should be something like:

curl -OL https://cache.ruby-lang.org/pub/ruby/snapshot/snapshot-master.tar.xz 

or https://cache.ruby-lang.org/pub/ruby/3.1/ruby-3.1.0-preview1.tar.xz and make sure no Ruby is installed on the system. I think this is the main issue and it does not really matter on the specific make command. But I'll try to come up with simple reproducer.

So the basic scenarios seem to work as expected (unexpectedly for me, sorry). But this is the failing scenario, where I am using expanded tarball:

$ history 1 cd /builddir/build/BUILD/ruby-3.1.0-74b58dd690/ 2 autoconf 3 ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info 4 make 5 make install DESTDIR=/tmp/local/ 6 history $ ruby -v sh: ruby: command not found 

Please note that these ./configure options are standard Fedora configuration options and they make the difference. With plain ./configure, the example would pass just fine. Also, as @darix pointed out, the plain make install also succeeds. But that is probably not really surprising, because in that case Ruby can already by found on the expected location prior gems are installed.

Note that we can reproduce the issue with --includedir configure option like that: ./configure --includedir=/usr/include && make -j && make install DESTDIR=/tmp/foo also on Ubuntu with git repo.
@nobu (Nobuyoshi Nakada) is checking.

Does this fix it?
https://github.com/ruby/ruby/pull/5318

vo.x (Vit Ondruch) wrote in #note-18:

No, unfortunately. Please see the attached Makefile. I think that the culprit is still the arch_hdrdir = $(extout)/include/$(arch)

Yes, it's the problem.
Could you show rbconfig.rb file too?

nobu (Nobuyoshi Nakada) wrote in #note-19:

Could you show rbconfig.rb file too?

Done

On Fri, 24 Dec 2021 11:44:21 +0000 (UTC)
"nobu (Nobuyoshi Nakada)" noreply@ruby-lang.org wrote:

Issue #18373 has been updated by nobu (Nobuyoshi Nakada).

Status changed from Closed to Open

As the patch seems to fix one situation at least anyway, committed
for now.

I just tested the latest snapshot (2021-12-25 01:00 AM) and it still
doesn't work in my packaging environment.

This is even more tricky then I initially thought, at least in the RPM context. The issue that RPM assumes there is "build" step, where everything is build and later "install" step, where everything is move into right location. Unfortunately, the way the files are currently installed mixes the two steps together, because the install steps builds the bundled gem binary extensions after the gem is installed. That in turn causes troubles with debug info stripping (which went through unnoticed for a while due to change in grep and therefore bug in RPM itself).

I starting to think that the right approach would be to build the binary extensions along the other Ruby modules and let the installer to only install the files.

Ah, now I found this git|bac6e8ca5d8f6bc45984d12ddad55d3d876d4324, so it used to be the case. I'll try to take a look what would the rbinstall.rb need to install also the binary extensions.

vo.x (Vit Ondruch) wrote in #note-25:

I have put together https://github.com/ruby/ruby/pull/5743 which should address this particular issue.

Not sure what $installed_list is good for, but the extensions as build currently are very likely not listed there. Just one another flaw of the current design.

vo.x (Vit Ondruch) wrote in #note-27:

vo.x (Vit Ondruch) wrote in #note-25:

I have put together https://github.com/ruby/ruby/pull/5743 which should address this particular issue.

I have updated the PR ^^ in a way that it seems to completely address my original issue and I am going to use it in Fedora package.