Update --ext=rust to support compiling the native extension from source

[karreiro]

What was the end-user or developer problem that led to this PR?

I needed to apply some tiny changes to the skeleton generated by the --ext=rust option to compile my native extension from source.

What is your fix for the problem, implemented in this PR?

When gem consumers install a gem with a native extension and their platform is not included the pre-compiled bundles, they need to compile it locally and rely on the extconf.rb file.

Referencing the extconf.rb file in the gemspec is required to make that work. We also need to include rb_sys as a dependency because the Rust extconf.rb depends on it.

Here's a demo of this change in action :)

Screen.Recording.2024-04-25.at.21.22.36.mov

[welcome]

Thanks for opening a pull request and helping make RubyGems and Bundler better! Someone from the RubyGems team will take a look at your pull request shortly and leave any feedback. Please make sure that your pull request has tests for any changes or added functionality.

We use GitHub Actions to test and make sure your change works functionally and uses acceptable conventions, you can review the current progress of GitHub Actions in the PR status window below.

If you have any questions or concerns that you wish to ask, feel free to leave a comment in this PR or join our #rubygems or #bundler channel on Slack.

For more information about contributing to the RubyGems project feel free to review our CONTRIBUTING guide

[karreiro]

👋 Hey @simi, thanks for chiming in on this PR. I've noticed the branch was outdated with the master, so I just updated it.

I'd appreciate your review when you have a moment :)

[simi]

Hello @karreiro and thanks for your PR. Can you please explain how to reproduce your errors? I just tried in latest bundler to create empty skeleton gem, pack it into gem and install with no problems. I have did following.

$ gem list | grep rb_sys  # ensure rb_sys is not installed already
$ bundle gem --ext=rust ryba > /dev/null # create empty gem
$ cd ryba # move into gem directory
$ cargo c # resolve the Cargo.lock somehow
$ git add Cargo.lock # add lock to the final gem, since gem files are collected from git by default
$ gem build --force # build gem, ignore TODO in gemspec
$ gem install ryba-0.1.0.gem # install local gem

[karreiro]

Hello @simi, thank you for reviewing this PR :)

Setup

To reproduce the error, let's create 3 ryba gems:

ryba1

• Run bundle gem --ext=rust ryba1 > /dev/null
• Run cd ryba1
• Run cargo c
• Run git add Cargo.lock
• Run gem build --force

ryba2

• Run bundle gem --ext=rust ryba2 > /dev/null
• Run cd ryba2
• Run cargo c
• Run git add Cargo.lock
• Update the ryba2.gemspec file to use spec.extensions = ["ext/ryba2/extconf.rb"] instead of spec.extensions = ["ext/ryba2/Cargo.toml"]
• Run gem build --force

ryba3

• Run bundle gem --ext=rust ryba3 > /dev/null
• Run cd ryba3
• Run cargo c
• Run git add Cargo.lock
• Update the ryba3.gemspec file to use spec.extensions = ["ext/ryba3/extconf.rb"] instead of spec.extensions = ["ext/ryba3/Cargo.toml"]
• Update the ryba3.gemspec file to include spec.add_dependency "rb_sys", "~> 0.9.91"
• Run gem build --force

Results

Now, we have 3 .gem files:

• ryba1-0.1.0.gem
• ryba2-0.1.0.gem
• ryba3-0.1.0.gem

Let's move to a machine without Rust installed now.

ryba1 (result)

When we run gem install ryba1-0.1.0.gem, we get an error like this:

Building native extensions. This could take a while...
ERROR:  Error installing ryba1-0.1.0.gem:
        ERROR: Failed to build gem native extension.

    cargo metadata failed No such file or directory - cargo

Gem files will remain installed in /opt/rubies/ruby-3.1.4/lib/ruby/gems/3.1.0/gems/ryba-0.1.0 for inspection.
Results logged to /opt/rubies/ruby-3.1.4/lib/ruby/gems/3.1.0/extensions/x86_64-linux/3.1.0/ryba-0.1.0/gem_make.out

ryba2 (result)

When we run gem install ryba2-0.1.0.gem, we get an error like this:

Building native extensions. This could take a while...
ERROR:  Error installing ryba2-0.1.0.gem:
        ERROR: Failed to build gem native extension.

    current directory: /opt/rubies/ruby-3.1.4/lib/ruby/gems/3.1.0/gems/ryba2-0.1.0/ext/ryba2
/opt/rubies/ruby-3.1.4/bin/ruby -I /opt/rubies/ruby-3.1.4/lib/ruby/site_ruby/3.1.0 extconf.rb
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/opt/rubies/ruby-3.1.4/bin/$(RUBY_BASE_NAME)
<internal:/opt/rubies/ruby-3.1.4/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require': cannot load such file -- rb_sys/mkmf (LoadError)
        from <internal:/opt/rubies/ruby-3.1.4/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:88:in `require'
        from extconf.rb:4:in `<main>'

extconf failed, exit code 1

Gem files will remain installed in /opt/rubies/ruby-3.1.4/lib/ruby/gems/3.1.0/gems/ryba2-0.1.0 for inspection.
Results logged to /opt/rubies/ruby-3.1.4/lib/ruby/gems/3.1.0/extensions/x86_64-linux/3.1.0/ryba2-0.1.0/gem_make.out

ryba3 (result)

When we run gem install ryba3-0.1.0.gem, it's going to work 🎉