-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Installing a Rust release #10181
Comments
As a package maintainer of fish in Fedora and an ardent user of fish shell, I would prefer that you keep using CMake. It's straightforward and already does what you need, and works well with packaging and shipping fish in distributions. |
Regrettably, there will be some work required for distributors; Fedora probably has it easiest as the imperative nature of RPM scripts means that including the macros that handle build dependencies and so on is straightforward. But at a minimum there will need to be additions made to handle the requirements of cargo being online and package builders being offline. CMake does not support much in the way of Rust by default; Corrosion is helping but requires either vendoring or online access to work properly and the method of getting CMake variables into the Rust build seems fragile to me. |
Fedora has a method of being able to tell Cargo to use a local index of offline crates packaged as RPMs, so that's not an issue. |
I maintain fish for Debian family (including Ubuntu etc). Feel free to at me here on GitHub, or reach out throught email. I'll have to check debcargo and get back to you soon. |
It's probably worth looking at an example of how rust programs are packaged today: https://gitlab.archlinux.org/archlinux/packaging/packages/ripgrep/-/blob/main/PKGBUILD This will
The package manager will then tar all that up and add the metadata to make a package, which it will then, on install, extract. For fish, a similar procedure would involve roughly, as far as I can tell:
Note that none of this would be done simply with This procedure seems pretty involved. It might be possible to create a script or xtask thing, but I'm not sure how well that works and how many people will try |
Continuing to use CMake is looking increasingly favourable. It would be worth looking into whether all of Corrosion is needed, or whether CMake could be taught to drive cargo without too much trouble. I don't think there's much to be gained from rewriting fish_test_helper from C++ to Rust, so work on this can start. |
I think trying to (force a) move from CMake right now isn’t necessary - there’s no rush to move away from a dependency we already have (and have already sorted out for release management purposes with the various package managers and distros). But, purely theoretically speaking, if we’re able to move all compilation-related stuff from CMake to build.rs over time (getting rid of corrosion in the process), we absolutely could get away with just a Makefile and build.rs without needing autotools or any other stuff that drives people away from Makefiles to CMake or whatever else in the first place (i.e. just Make as a cross-platform, lightweight, always available; familiar, and distro-friendly task runner rather than an actual build system). In that context, I don’t see the benefit of xtask over a Makefile (and see other drawbacks). |
Corrosion handles a lot for you that you probably don't want to bother with, for example integrating how CMake handles cross compilation with how Cargo handles it. I'd caution against calling Cargo directly from CMake without Corrosion. |
There's a major limitation with build.rs that you'd bump into: there's intentionally no way for build.rs to say "generate this file and put it here" and have an external tool know where "here" is. There's a way to hack around this though: have an external tool (Make, CMake, a shell script, or whatever) set an environment variable telling build.rs where to put generated files when calling |
If I may chime in:
My personal decision tree here:
Additional thing here is that, as far as I understand, fish also depends on native libraries (pcre & ncurses). I think the biggest benefit that CMake provides is actually finding those libraries and allowing the packager to substitute them in a nice way. Cargo, through custom build.rs, of course allows the packager to control these as well, but there isn't a declarative, standard way to do so. Eg, different rust projects might end up using different mechanisms for customization, for example, input environment variables vs pkg-config. |
I broadly agree with your conclusion (specifically the first), but I think I disagree in some of the details. I know you've done a bit to popularise the "xtask" pattern, where you add another rust binary that is run via cargo to do stuff, and that's nice for "simpler" projects, but I'm afraid the complexity we would need there makes my head spin. E.g. zellij has what I believe classifies as a "big" xtask, and they copy three asset files, one of which is a man page. For comparison we have about the same amount of lines of .fish as .rs, and we have a quarter of that in docs that we want as both man pages and html, plus auxilliary files like pkg-config. I don't see any handling of things like DESTDIR/PREFIX/DATADIR (which fish uses at runtime to find out where the functions and stuff are). I have no idea how cross-compilation would work, especially when you have e.g. the docs depending on a working fish_indent on the generating system (tho to be fair I don't really know how we handle that today). They also end up shelling out in a bunch of places anyway, which... well maybe I'm a shellscript snob, but that always felt nicer in a shellscript to me.
Rust is more elegant generally than CMake, but CMake is the better build tool. And cmake is cross-platform enough that it runs everywhere we currently support fish, and we're not planning on supporting Windows. And it being "ugly" doesn't matter so much when most of the work writing any cmake script is already done.
That's quickly looking better and better. I think we can get to a point where you can I don't see a winning argument for switching back to makefiles - the cmake dependency isn't a problem in practice (especially not for distributions) and we switched for a reason. If anyone has a compelling argument I would like to hear it, and what make features specifically we would want. Part of would prefer writing a shellscript over a makefile but that's probably unpopular. And of course: If something happens in cargo or as an extension to it, we can always revisit this decision and pick a nicer, rustier, build tool. |
Starting with what I said before: we already have CMake and removing it doesn’t especially further the particular interests of the project in the same way that removing C++ from the project did, and there is no shame in making one (or many) release with a bastardized build system inspired by the project’s C/C++ roots but with the modernized rust toolchain also used in parts. That said:
Fully agreed, but
I want to get here too.
If we get to the point where we can build all the binaries, correctly configured, without CMake doing more than providing the parameters (what and where), then it’s being a better build tool is irrelevant. Which nullifies the advantage of CMake over Makefiles or over a rust runner. I am - and always have been - partial to the beauty, simplicity, and universality of (Look at how non-packagers can use Make now to drive fish’s CMake and the C++ build toolchain without knowing a thing about how the build system actually works, and it gracefully-ish handles the missing baseline components of the build system by telling users what they need to install to get started.) |
Well, no, because: CMake already exists. We already have it. Most of the work we would have to do on it is to remove stuff, not add anything. (we would have to add to build.rs if we want Plus, unlike you, I've never really liked makefiles, and we've always disagreed on how unknown cmake really is (I don't think running But if you want to write a Makefile, and it manages to be nicer than our current cmake system, we can of course merge it. I would want it to have rough feature-parity tho. In particular it should support detection of curses and pcre, the output variables (DESTDIR/PREFIX etc), installing, out-of-tree builds and building with/without localization or docs. Cross-compilation would also be great to have (but again I've never tested our cross-compilation with cmake, I'm not sure if people actually use it). Ideally it wouldn't require autotools. (side note: it would be really cool if we found a rust-replacement for curses - we mostly use it to read terminfo, which as I understand it should be possible without linking to libcurses) |
This was my initial approach and I think there's a committed version of the riir branch that does just that, but unfortunately it turned out that many distros no longer ship a terminfo/termcap database or ship an out-of-date one. ncurses went from reading that to hard-coding its own equivalent in the library itself at some point, killing that as an option :( edit: not in the riir branch, but its own repo here: https://github.com/mqudsi/termcap |
So it appears to be looking like "hollowed-out cmake, unless something better materializes". In that case we would have to figure out how to make it easy to package. Looking at arch's rust package guidelines I see recommendations to run Is that something we can accomodate? I'm assuming How would this work on systems where the builder has no internet access? |
In two cases:
|
Alright, some changes since we last updated this:
So building is now a lot easier than it was. The next part I admit is a bit spicy: I would like to support installation via I believe it is possible to get something usable (if not perfect), and I believe it is a nice, cross-platform way to install fish especially for people already used to rust. This requires doing a lot of work via build.rs instead of cmake, and I believe it is possible to drop cmake almost entirely. To that end I've been working on my fish-installer branch. What that currently offers is this:
I will admit this is not perfect. It is unfortunate that there is this second step of running fish_installer. We could technically embed the asset files in the fish binary and either extract them automatically (which is a bit naughty) or just read them from inside the binary (note: rust-embed supports reading from the filesystem in debug-mode, so you don't need to rebuild fish every time you change a script during development). The main downside of that is a binary that is two to three times as large. There are also some missing bits, like the html docs, the pkg-config file, the vendor directories (they're bound to DATADIR so this fish would never read e.g. /usr/share/fish/vendor_confdirs - I'm unsure if this makes sense in general or if we should change it). I believe it is possible to get the manpages and translations included as well by generating them in build.rs. HTML docs are basically impossible because they rely on fish_indent for highlighting and build.rs runs before that is built, there is no other place to generate the docs. At that point, cmake is basically relegated to:
At which point replacing it with an "install.sh" shellscript becomes tractable. (and yes, my preference is for a shellscript, since cargo handles the actual build and so make-et-al's main advantage of skipping work does not come into play) |
I'm just not convinced that the target market (people with a Rust toolchain installed but not using Nix or Homebrew which make installation very easy) is that big, but I'm not hugely opposed to any of the steps you've planned. I don't want to underestimate how much distribution packages hate custom workflows (cf #10181 (comment)) but if it's as simple as |
Note that this also enables creating statically-linked self-installable binaries. I have, right now, a This would remove the need for appimages, and it's enabled by what enables
Alternatively we can keep cmake for them, it just won't do a lot. |
Custom workflows are inevitable as long as Cargo provides no canonical solution how to do this. Using CMake for installation is something packagers know how to handle, but in a sense it's still a strange custom workflow for a Rust program. |
By "custom" zanchey was, of course, referring to something alien to packagers. CMake would not count, because packagers know how to drive it. (unless we required you to pass things you typically don't) |
This issue is to discuss how to drive the installation process of the rewritten-in-rust fish.
cargo install
is not suitable for a complete, working fish install. The following tasks are required:It may also:
Some options:
cargo xtask
patterns. Major downside is that there's a lot of things that Make/CMake provide for "free" that would need re-implementation in Rust.https://github.com/axodotdev/cargo-dist is cool, but does not really solve the problem of making a source tarball that can then be built by distrbutions.
The text was updated successfully, but these errors were encountered: