aboutsummaryrefslogtreecommitdiff
path: root/scripts
diff options
context:
space:
mode:
authorLinus Torvalds2023-08-29 08:19:46 -0700
committerLinus Torvalds2023-08-29 08:19:46 -0700
commita031fe8d1d32898582e36ccbffa9847d16f67aa2 (patch)
treebb097e00fcf06c92efffe95e48163e5658d527fc /scripts
parentf2586d921cea4feeddd1cc5ee3495700540dba8f (diff)
parent4af84c6a85c63bec24611e46bb3de2c0a6602a51 (diff)
Merge tag 'rust-6.6' of https://github.com/Rust-for-Linux/linux
Pull rust updates from Miguel Ojeda: "In terms of lines, most changes this time are on the pinned-init API and infrastructure. While we have a Rust version upgrade, and thus a bunch of changes from the vendored 'alloc' crate as usual, this time those do not account for many lines. Toolchain and infrastructure: - Upgrade to Rust 1.71.1. This is the second such upgrade, which is a smaller jump compared to the last time. This version allows us to remove the '__rust_*' allocator functions -- the compiler now generates them as expected, thus now our 'KernelAllocator' is used. It also introduces the 'offset_of!' macro in the standard library (as an unstable feature) which we will need soon. So far, we were using a declarative macro as a prerequisite in some not-yet-landed patch series, which did not support sub-fields (i.e. nested structs): #[repr(C)] struct S { a: u16, b: (u8, u8), } assert_eq!(offset_of!(S, b.1), 3); - Upgrade to bindgen 0.65.1. This is the first time we upgrade its version. Given it is a fairly big jump, it comes with a fair number of improvements/changes that affect us, such as a fix needed to support LLVM 16 as well as proper support for '__noreturn' C functions, which are now mapped to return the '!' type in Rust: void __noreturn f(void); // C pub fn f() -> !; // Rust - 'scripts/rust_is_available.sh' improvements and fixes. This series takes care of all the issues known so far and adds a few new checks to cover for even more cases, plus adds some more help texts. All this together will hopefully make problematic setups easier to identify and to be solved by users building the kernel. In addition, it adds a test suite which covers all branches of the shell script, as well as tests for the issues found so far. - Support rust-analyzer for out-of-tree modules too. - Give 'cfg's to rust-analyzer for the 'core' and 'alloc' crates. - Drop 'scripts/is_rust_module.sh' since it is not needed anymore. Macros crate: - New 'paste!' proc macro. This macro is a more flexible version of 'concat_idents!': it allows the resulting identifier to be used to declare new items and it allows to transform the identifiers before concatenating them, e.g. let x_1 = 42; paste!(let [<x _2>] = [<x _1>];); assert!(x_1 == x_2); The macro is then used for several of the pinned-init API changes in this pull. Pinned-init API: - Make '#[pin_data]' compatible with conditional compilation of fields, allowing to write code like: #[pin_data] pub struct Foo { #[cfg(CONFIG_BAR)] a: Bar, #[cfg(not(CONFIG_BAR))] a: Baz, } - New '#[derive(Zeroable)]' proc macro for the 'Zeroable' trait, which allows 'unsafe' implementations for structs where every field implements the 'Zeroable' trait, e.g.: #[derive(Zeroable)] pub struct DriverData { id: i64, buf_ptr: *mut u8, len: usize, } - Add '..Zeroable::zeroed()' syntax to the 'pin_init!' macro for zeroing all other fields, e.g.: pin_init!(Buf { buf: [1; 64], ..Zeroable::zeroed() }); - New '{,pin_}init_array_from_fn()' functions to create array initializers given a generator function, e.g.: let b: Box<[usize; 1_000]> = Box::init::<Error>( init_array_from_fn(|i| i) ).unwrap(); assert_eq!(b.len(), 1_000); assert_eq!(b[123], 123); - New '{,pin_}chain' methods for '{,Pin}Init<T, E>' that allow to execute a closure on the value directly after initialization, e.g.: let foo = init!(Foo { buf <- init::zeroed() }).chain(|foo| { foo.setup(); Ok(()) }); - Support arbitrary paths in init macros, instead of just identifiers and generic types. - Implement the 'Zeroable' trait for the 'UnsafeCell<T>' and 'Opaque<T>' types. - Make initializer values inaccessible after initialization. - Make guards in the init macros hygienic. 'allocator' module: - Use 'krealloc_aligned()' in 'KernelAllocator::alloc' preventing misaligned allocations when the Rust 1.71.1 upgrade is applied later in this pull. The equivalent fix for the previous compiler version (where 'KernelAllocator' is not yet used) was merged into 6.5 already, which added the 'krealloc_aligned()' function used here. - Implement 'KernelAllocator::{realloc, alloc_zeroed}' for performance, using 'krealloc_aligned()' too, which forwards the call to the C API. 'types' module: - Make 'Opaque' be '!Unpin', removing the need to add a 'PhantomPinned' field to Rust structs that contain C structs which must not be moved. - Make 'Opaque' use 'UnsafeCell' as the outer type, rather than inner. Documentation: - Suggest obtaining the source code of the Rust's 'core' library using the tarball instead of the repository. MAINTAINERS: - Andreas and Alice, from Samsung and Google respectively, are joining as reviewers of the "RUST" entry. As well as a few other minor changes and cleanups" * tag 'rust-6.6' of https://github.com/Rust-for-Linux/linux: (42 commits) rust: init: update expanded macro explanation rust: init: add `{pin_}chain` functions to `{Pin}Init<T, E>` rust: init: make `PinInit<T, E>` a supertrait of `Init<T, E>` rust: init: implement `Zeroable` for `UnsafeCell<T>` and `Opaque<T>` rust: init: add support for arbitrary paths in init macros rust: init: add functions to create array initializers rust: init: add `..Zeroable::zeroed()` syntax for zeroing all missing fields rust: init: make initializer values inaccessible after initializing rust: init: wrap type checking struct initializers in a closure rust: init: make guards in the init macros hygienic rust: add derive macro for `Zeroable` rust: init: make `#[pin_data]` compatible with conditional compilation of fields rust: init: consolidate init macros docs: rust: clarify what 'rustup override' does docs: rust: update instructions for obtaining 'core' source docs: rust: add command line to rust-analyzer section scripts: generate_rust_analyzer: provide `cfg`s for `core` and `alloc` rust: bindgen: upgrade to 0.65.1 rust: enable `no_mangle_with_rust_abi` Clippy lint rust: upgrade to Rust 1.71.1 ...
Diffstat (limited to 'scripts')
-rw-r--r--scripts/Makefile.modfinal2
-rwxr-xr-xscripts/generate_rust_analyzer.py39
-rwxr-xr-xscripts/is_rust_module.sh16
-rwxr-xr-xscripts/min-tool-version.sh4
-rwxr-xr-xscripts/rust_is_available.sh233
-rwxr-xr-xscripts/rust_is_available_test.py346
6 files changed, 545 insertions, 95 deletions
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index fc19f67039bd..b3a6aa8fbe8c 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -41,8 +41,6 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
- elif [ -n "$(CONFIG_RUST)" ] && $(srctree)/scripts/is_rust_module.sh $@; then \
- printf "Skipping BTF generation for %s because it's a Rust module\n" $@ 1>&2; \
else \
LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \
$(RESOLVE_BTFIDS) -b vmlinux $@; \
diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index 946e250c1b2a..fc52bc41d3e7 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -6,10 +6,19 @@
import argparse
import json
import logging
+import os
import pathlib
import sys
-def generate_crates(srctree, objtree, sysroot_src):
+def args_crates_cfgs(cfgs):
+ crates_cfgs = {}
+ for cfg in cfgs:
+ crate, vals = cfg.split("=", 1)
+ crates_cfgs[crate] = vals.replace("--cfg", "").split()
+
+ return crates_cfgs
+
+def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs):
# Generate the configuration list.
cfg = []
with open(objtree / "include" / "generated" / "rustc_cfg") as fd:
@@ -23,6 +32,7 @@ def generate_crates(srctree, objtree, sysroot_src):
# Avoid O(n^2) iterations by keeping a map of indexes.
crates = []
crates_indexes = {}
+ crates_cfgs = args_crates_cfgs(cfgs)
def append_crate(display_name, root_module, deps, cfg=[], is_workspace_member=True, is_proc_macro=False):
crates_indexes[display_name] = len(crates)
@@ -44,6 +54,7 @@ def generate_crates(srctree, objtree, sysroot_src):
"core",
sysroot_src / "core" / "src" / "lib.rs",
[],
+ cfg=crates_cfgs.get("core", []),
is_workspace_member=False,
)
@@ -57,6 +68,7 @@ def generate_crates(srctree, objtree, sysroot_src):
"alloc",
srctree / "rust" / "alloc" / "lib.rs",
["core", "compiler_builtins"],
+ cfg=crates_cfgs.get("alloc", []),
)
append_crate(
@@ -65,7 +77,7 @@ def generate_crates(srctree, objtree, sysroot_src):
[],
is_proc_macro=True,
)
- crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so"
+ crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so"
append_crate(
"build_error",
@@ -95,19 +107,26 @@ def generate_crates(srctree, objtree, sysroot_src):
"exclude_dirs": [],
}
+ def is_root_crate(build_file, target):
+ try:
+ return f"{target}.o" in open(build_file).read()
+ except FileNotFoundError:
+ return False
+
# Then, the rest outside of `rust/`.
#
# We explicitly mention the top-level folders we want to cover.
- for folder in ("samples", "drivers"):
- for path in (srctree / folder).rglob("*.rs"):
+ extra_dirs = map(lambda dir: srctree / dir, ("samples", "drivers"))
+ if external_src is not None:
+ extra_dirs = [external_src]
+ for folder in extra_dirs:
+ for path in folder.rglob("*.rs"):
logging.info("Checking %s", path)
name = path.name.replace(".rs", "")
# Skip those that are not crate roots.
- try:
- if f"{name}.o" not in open(path.parent / "Makefile").read():
- continue
- except FileNotFoundError:
+ if not is_root_crate(path.parent / "Makefile", name) and \
+ not is_root_crate(path.parent / "Kbuild", name):
continue
logging.info("Adding %s", name)
@@ -123,9 +142,11 @@ def generate_crates(srctree, objtree, sysroot_src):
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--verbose', '-v', action='store_true')
+ parser.add_argument('--cfgs', action='append', default=[])
parser.add_argument("srctree", type=pathlib.Path)
parser.add_argument("objtree", type=pathlib.Path)
parser.add_argument("sysroot_src", type=pathlib.Path)
+ parser.add_argument("exttree", type=pathlib.Path, nargs="?")
args = parser.parse_args()
logging.basicConfig(
@@ -134,7 +155,7 @@ def main():
)
rust_project = {
- "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src),
+ "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree, args.cfgs),
"sysroot_src": str(args.sysroot_src),
}
diff --git a/scripts/is_rust_module.sh b/scripts/is_rust_module.sh
deleted file mode 100755
index 464761a7cf7f..000000000000
--- a/scripts/is_rust_module.sh
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-#
-# is_rust_module.sh module.ko
-#
-# Returns `0` if `module.ko` is a Rust module, `1` otherwise.
-
-set -e
-
-# Using the `16_` prefix ensures other symbols with the same substring
-# are not picked up (even if it would be unlikely). The last part is
-# used just in case LLVM decides to use the `.` suffix.
-#
-# In the future, checking for the `.comment` section may be another
-# option, see https://github.com/rust-lang/rust/pull/97550.
-${NM} "$*" | grep -qE '^[0-9a-fA-F]+ [Rr] _R[^[:space:]]+16___IS_RUST_MODULE[^[:space:]]*$'
diff --git a/scripts/min-tool-version.sh b/scripts/min-tool-version.sh
index 2ade63149466..d65ab8bfeaf4 100755
--- a/scripts/min-tool-version.sh
+++ b/scripts/min-tool-version.sh
@@ -31,10 +31,10 @@ llvm)
fi
;;
rustc)
- echo 1.68.2
+ echo 1.71.1
;;
bindgen)
- echo 0.56.0
+ echo 0.65.1
;;
*)
echo "$1: unknown tool" >&2
diff --git a/scripts/rust_is_available.sh b/scripts/rust_is_available.sh
index aebbf1913970..117018946b57 100755
--- a/scripts/rust_is_available.sh
+++ b/scripts/rust_is_available.sh
@@ -2,8 +2,6 @@
# SPDX-License-Identifier: GPL-2.0
#
# Tests whether a suitable Rust toolchain is available.
-#
-# Pass `-v` for human output and more checks (as warnings).
set -e
@@ -21,102 +19,208 @@ get_canonical_version()
echo $((100000 * $1 + 100 * $2 + $3))
}
+# Print a reference to the Quick Start guide in the documentation.
+print_docs_reference()
+{
+ echo >&2 "***"
+ echo >&2 "*** Please see Documentation/rust/quick-start.rst for details"
+ echo >&2 "*** on how to set up the Rust support."
+ echo >&2 "***"
+}
+
+# Print an explanation about the fact that the script is meant to be called from Kbuild.
+print_kbuild_explanation()
+{
+ echo >&2 "***"
+ echo >&2 "*** This script is intended to be called from Kbuild."
+ echo >&2 "*** Please use the 'rustavailable' target to call it instead."
+ echo >&2 "*** Otherwise, the results may not be meaningful."
+ exit 1
+}
+
+# If the script fails for any reason, or if there was any warning, then
+# print a reference to the documentation on exit.
+warning=0
+trap 'if [ $? -ne 0 ] || [ $warning -ne 0 ]; then print_docs_reference; fi' EXIT
+
+# Check that the expected environment variables are set.
+if [ -z "${RUSTC+x}" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Environment variable 'RUSTC' is not set."
+ print_kbuild_explanation
+fi
+
+if [ -z "${BINDGEN+x}" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Environment variable 'BINDGEN' is not set."
+ print_kbuild_explanation
+fi
+
+if [ -z "${CC+x}" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Environment variable 'CC' is not set."
+ print_kbuild_explanation
+fi
+
# Check that the Rust compiler exists.
if ! command -v "$RUSTC" >/dev/null; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** Rust compiler '$RUSTC' could not be found."
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** Rust compiler '$RUSTC' could not be found."
+ echo >&2 "***"
exit 1
fi
# Check that the Rust bindings generator exists.
if ! command -v "$BINDGEN" >/dev/null; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** Rust bindings generator '$BINDGEN' could not be found."
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** Rust bindings generator '$BINDGEN' could not be found."
+ echo >&2 "***"
exit 1
fi
# Check that the Rust compiler version is suitable.
#
# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
+rust_compiler_output=$( \
+ LC_ALL=C "$RUSTC" --version 2>/dev/null
+) || rust_compiler_code=$?
+if [ -n "$rust_compiler_code" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$RUSTC' to check the Rust compiler version failed with"
+ echo >&2 "*** code $rust_compiler_code. See output and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$rust_compiler_output"
+ echo >&2 "***"
+ exit 1
+fi
rust_compiler_version=$( \
- LC_ALL=C "$RUSTC" --version 2>/dev/null \
- | head -n 1 \
- | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+ echo "$rust_compiler_output" \
+ | sed -nE '1s:.*rustc ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
)
+if [ -z "$rust_compiler_version" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$RUSTC' to check the Rust compiler version did not return"
+ echo >&2 "*** an expected output. See output and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$rust_compiler_output"
+ echo >&2 "***"
+ exit 1
+fi
rust_compiler_min_version=$($min_tool_version rustc)
rust_compiler_cversion=$(get_canonical_version $rust_compiler_version)
rust_compiler_min_cversion=$(get_canonical_version $rust_compiler_min_version)
if [ "$rust_compiler_cversion" -lt "$rust_compiler_min_cversion" ]; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** Rust compiler '$RUSTC' is too old."
- echo >&2 "*** Your version: $rust_compiler_version"
- echo >&2 "*** Minimum version: $rust_compiler_min_version"
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** Rust compiler '$RUSTC' is too old."
+ echo >&2 "*** Your version: $rust_compiler_version"
+ echo >&2 "*** Minimum version: $rust_compiler_min_version"
+ echo >&2 "***"
exit 1
fi
-if [ "$1" = -v ] && [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
+if [ "$rust_compiler_cversion" -gt "$rust_compiler_min_cversion" ]; then
echo >&2 "***"
echo >&2 "*** Rust compiler '$RUSTC' is too new. This may or may not work."
echo >&2 "*** Your version: $rust_compiler_version"
echo >&2 "*** Expected version: $rust_compiler_min_version"
echo >&2 "***"
+ warning=1
fi
# Check that the Rust bindings generator is suitable.
#
# Non-stable and distributions' versions may have a version suffix, e.g. `-dev`.
+rust_bindings_generator_output=$( \
+ LC_ALL=C "$BINDGEN" --version 2>/dev/null
+) || rust_bindings_generator_code=$?
+if [ -n "$rust_bindings_generator_code" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$BINDGEN' to check the Rust bindings generator version failed with"
+ echo >&2 "*** code $rust_bindings_generator_code. See output and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$rust_bindings_generator_output"
+ echo >&2 "***"
+ exit 1
+fi
rust_bindings_generator_version=$( \
- LC_ALL=C "$BINDGEN" --version 2>/dev/null \
- | head -n 1 \
- | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
+ echo "$rust_bindings_generator_output" \
+ | sed -nE '1s:.*bindgen ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
)
+if [ -z "$rust_bindings_generator_version" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$BINDGEN' to check the bindings generator version did not return"
+ echo >&2 "*** an expected output. See output and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$rust_bindings_generator_output"
+ echo >&2 "***"
+ exit 1
+fi
rust_bindings_generator_min_version=$($min_tool_version bindgen)
rust_bindings_generator_cversion=$(get_canonical_version $rust_bindings_generator_version)
rust_bindings_generator_min_cversion=$(get_canonical_version $rust_bindings_generator_min_version)
if [ "$rust_bindings_generator_cversion" -lt "$rust_bindings_generator_min_cversion" ]; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** Rust bindings generator '$BINDGEN' is too old."
- echo >&2 "*** Your version: $rust_bindings_generator_version"
- echo >&2 "*** Minimum version: $rust_bindings_generator_min_version"
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** Rust bindings generator '$BINDGEN' is too old."
+ echo >&2 "*** Your version: $rust_bindings_generator_version"
+ echo >&2 "*** Minimum version: $rust_bindings_generator_min_version"
+ echo >&2 "***"
exit 1
fi
-if [ "$1" = -v ] && [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cversion" ]; then
+if [ "$rust_bindings_generator_cversion" -gt "$rust_bindings_generator_min_cversion" ]; then
echo >&2 "***"
echo >&2 "*** Rust bindings generator '$BINDGEN' is too new. This may or may not work."
echo >&2 "*** Your version: $rust_bindings_generator_version"
echo >&2 "*** Expected version: $rust_bindings_generator_min_version"
echo >&2 "***"
+ warning=1
fi
# Check that the `libclang` used by the Rust bindings generator is suitable.
+#
+# In order to do that, first invoke `bindgen` to get the `libclang` version
+# found by `bindgen`. This step may already fail if, for instance, `libclang`
+# is not found, thus inform the user in such a case.
+bindgen_libclang_output=$( \
+ LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null
+) || bindgen_libclang_code=$?
+if [ -n "$bindgen_libclang_code" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$BINDGEN' to check the libclang version (used by the Rust"
+ echo >&2 "*** bindings generator) failed with code $bindgen_libclang_code. This may be caused by"
+ echo >&2 "*** a failure to locate libclang. See output and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$bindgen_libclang_output"
+ echo >&2 "***"
+ exit 1
+fi
+
+# `bindgen` returned successfully, thus use the output to check that the version
+# of the `libclang` found by the Rust bindings generator is suitable.
+#
+# Unlike other version checks, note that this one does not necessarily appear
+# in the first line of the output, thus no `sed` address is provided.
bindgen_libclang_version=$( \
- LC_ALL=C "$BINDGEN" $(dirname $0)/rust_is_available_bindgen_libclang.h 2>&1 >/dev/null \
- | grep -F 'clang version ' \
- | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' \
- | head -n 1 \
+ echo "$bindgen_libclang_output" \
+ | sed -nE 's:.*clang version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
)
+if [ -z "$bindgen_libclang_version" ]; then
+ echo >&2 "***"
+ echo >&2 "*** Running '$BINDGEN' to check the libclang version (used by the Rust"
+ echo >&2 "*** bindings generator) did not return an expected output. See output"
+ echo >&2 "*** and docs below for details:"
+ echo >&2 "***"
+ echo >&2 "$bindgen_libclang_output"
+ echo >&2 "***"
+ exit 1
+fi
bindgen_libclang_min_version=$($min_tool_version llvm)
bindgen_libclang_cversion=$(get_canonical_version $bindgen_libclang_version)
bindgen_libclang_min_cversion=$(get_canonical_version $bindgen_libclang_min_version)
if [ "$bindgen_libclang_cversion" -lt "$bindgen_libclang_min_cversion" ]; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN') is too old."
- echo >&2 "*** Your version: $bindgen_libclang_version"
- echo >&2 "*** Minimum version: $bindgen_libclang_min_version"
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN') is too old."
+ echo >&2 "*** Your version: $bindgen_libclang_version"
+ echo >&2 "*** Minimum version: $bindgen_libclang_min_version"
+ echo >&2 "***"
exit 1
fi
@@ -125,21 +229,20 @@ fi
#
# In the future, we might be able to perform a full version check, see
# https://github.com/rust-lang/rust-bindgen/issues/2138.
-if [ "$1" = -v ]; then
- cc_name=$($(dirname $0)/cc-version.sh "$CC" | cut -f1 -d' ')
- if [ "$cc_name" = Clang ]; then
- clang_version=$( \
- LC_ALL=C "$CC" --version 2>/dev/null \
- | sed -nE '1s:.*version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
- )
- if [ "$clang_version" != "$bindgen_libclang_version" ]; then
- echo >&2 "***"
- echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN')"
- echo >&2 "*** version does not match Clang's. This may be a problem."
- echo >&2 "*** libclang version: $bindgen_libclang_version"
- echo >&2 "*** Clang version: $clang_version"
- echo >&2 "***"
- fi
+cc_name=$($(dirname $0)/cc-version.sh $CC | cut -f1 -d' ')
+if [ "$cc_name" = Clang ]; then
+ clang_version=$( \
+ LC_ALL=C $CC --version 2>/dev/null \
+ | sed -nE '1s:.*version ([0-9]+\.[0-9]+\.[0-9]+).*:\1:p'
+ )
+ if [ "$clang_version" != "$bindgen_libclang_version" ]; then
+ echo >&2 "***"
+ echo >&2 "*** libclang (used by the Rust bindings generator '$BINDGEN')"
+ echo >&2 "*** version does not match Clang's. This may be a problem."
+ echo >&2 "*** libclang version: $bindgen_libclang_version"
+ echo >&2 "*** Clang version: $clang_version"
+ echo >&2 "***"
+ warning=1
fi
fi
@@ -150,11 +253,9 @@ rustc_sysroot=$("$RUSTC" $KRUSTFLAGS --print sysroot)
rustc_src=${RUST_LIB_SRC:-"$rustc_sysroot/lib/rustlib/src/rust/library"}
rustc_src_core="$rustc_src/core/src/lib.rs"
if [ ! -e "$rustc_src_core" ]; then
- if [ "$1" = -v ]; then
- echo >&2 "***"
- echo >&2 "*** Source code for the 'core' standard library could not be found"
- echo >&2 "*** at '$rustc_src_core'."
- echo >&2 "***"
- fi
+ echo >&2 "***"
+ echo >&2 "*** Source code for the 'core' standard library could not be found"
+ echo >&2 "*** at '$rustc_src_core'."
+ echo >&2 "***"
exit 1
fi
diff --git a/scripts/rust_is_available_test.py b/scripts/rust_is_available_test.py
new file mode 100755
index 000000000000..57613fe5ed75
--- /dev/null
+++ b/scripts/rust_is_available_test.py
@@ -0,0 +1,346 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+"""Tests the `rust_is_available.sh` script.
+
+Some of the tests require the real programs to be available in `$PATH`
+under their canonical name (and with the expected versions).
+"""
+
+import enum
+import os
+import pathlib
+import stat
+import subprocess
+import tempfile
+import unittest
+
+class TestRustIsAvailable(unittest.TestCase):
+ @enum.unique
+ class Expected(enum.Enum):
+ SUCCESS = enum.auto()
+ SUCCESS_WITH_WARNINGS = enum.auto()
+ SUCCESS_WITH_EXTRA_OUTPUT = enum.auto()
+ FAILURE = enum.auto()
+
+ @classmethod
+ def generate_executable(cls, content):
+ path = pathlib.Path(cls.tempdir.name)
+ name = str(len(tuple(path.iterdir())))
+ path = path / name
+ with open(path, "w") as file_:
+ file_.write(content)
+ os.chmod(path, os.stat(path).st_mode | stat.S_IXUSR)
+ return path
+
+ @classmethod
+ def generate_clang(cls, stdout):
+ return cls.generate_executable(f"""#!/usr/bin/env python3
+import sys
+if "-E" in " ".join(sys.argv):
+ print({repr("Clang " + " ".join(cls.llvm_default_version.split(" ")))})
+else:
+ print({repr(stdout)})
+""")
+
+ @classmethod
+ def generate_rustc(cls, stdout):
+ return cls.generate_executable(f"""#!/usr/bin/env python3
+import sys
+if "--print sysroot" in " ".join(sys.argv):
+ print({repr(cls.rust_default_sysroot)})
+else:
+ print({repr(stdout)})
+""")
+
+ @classmethod
+ def generate_bindgen(cls, version_stdout, libclang_stderr):
+ return cls.generate_executable(f"""#!/usr/bin/env python3
+import sys
+if "rust_is_available_bindgen_libclang.h" in " ".join(sys.argv):
+ print({repr(libclang_stderr)}, file=sys.stderr)
+else:
+ print({repr(version_stdout)})
+""")
+
+ @classmethod
+ def generate_bindgen_version(cls, stdout):
+ return cls.generate_bindgen(stdout, cls.bindgen_default_bindgen_libclang_stderr)
+
+ @classmethod
+ def generate_bindgen_libclang(cls, stderr):
+ return cls.generate_bindgen(cls.bindgen_default_bindgen_version_stdout, stderr)
+
+ @classmethod
+ def setUpClass(cls):
+ cls.tempdir = tempfile.TemporaryDirectory()
+
+ cls.missing = pathlib.Path(cls.tempdir.name) / "missing"
+
+ cls.nonexecutable = pathlib.Path(cls.tempdir.name) / "nonexecutable"
+ with open(cls.nonexecutable, "w") as file_:
+ file_.write("nonexecutable")
+
+ cls.unexpected_binary = "true"
+
+ cls.rustc_default_version = subprocess.check_output(("scripts/min-tool-version.sh", "rustc")).decode().strip()
+ cls.bindgen_default_version = subprocess.check_output(("scripts/min-tool-version.sh", "bindgen")).decode().strip()
+ cls.llvm_default_version = subprocess.check_output(("scripts/min-tool-version.sh", "llvm")).decode().strip()
+ cls.rust_default_sysroot = subprocess.check_output(("rustc", "--print", "sysroot")).decode().strip()
+
+ cls.bindgen_default_bindgen_version_stdout = f"bindgen {cls.bindgen_default_version}"
+ cls.bindgen_default_bindgen_libclang_stderr = f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {cls.llvm_default_version} [-W#pragma-messages], err: false"
+
+ cls.default_rustc = cls.generate_rustc(f"rustc {cls.rustc_default_version}")
+ cls.default_bindgen = cls.generate_bindgen(cls.bindgen_default_bindgen_version_stdout, cls.bindgen_default_bindgen_libclang_stderr)
+ cls.default_cc = cls.generate_clang(f"clang version {cls.llvm_default_version}")
+
+ def run_script(self, expected, override_env):
+ env = {
+ "RUSTC": self.default_rustc,
+ "BINDGEN": self.default_bindgen,
+ "CC": self.default_cc,
+ }
+
+ for key, value in override_env.items():
+ if value is None:
+ del env[key]
+ continue
+ env[key] = value
+
+ result = subprocess.run("scripts/rust_is_available.sh", env=env, capture_output=True)
+
+ # The script should never output anything to `stdout`.
+ self.assertEqual(result.stdout, b"")
+
+ if expected == self.Expected.SUCCESS:
+ # When expecting a success, the script should return 0
+ # and it should not output anything to `stderr`.
+ self.assertEqual(result.returncode, 0)
+ self.assertEqual(result.stderr, b"")
+ elif expected == self.Expected.SUCCESS_WITH_EXTRA_OUTPUT:
+ # When expecting a success with extra output (that is not warnings,
+ # which is the common case), the script should return 0 and it
+ # should output at least something to `stderr` (the output should
+ # be checked further by the test).
+ self.assertEqual(result.returncode, 0)
+ self.assertNotEqual(result.stderr, b"")
+ elif expected == self.Expected.SUCCESS_WITH_WARNINGS:
+ # When expecting a success with warnings, the script should return 0
+ # and it should output at least the instructions to `stderr`.
+ self.assertEqual(result.returncode, 0)
+ self.assertIn(b"Please see Documentation/rust/quick-start.rst for details", result.stderr)
+ else:
+ # When expecting a failure, the script should return non-0
+ # and it should output at least the instructions to `stderr`.
+ self.assertNotEqual(result.returncode, 0)
+ self.assertIn(b"Please see Documentation/rust/quick-start.rst for details", result.stderr)
+
+ # The output will generally be UTF-8 (i.e. unless the user has
+ # put strange values in the environment).
+ result.stderr = result.stderr.decode()
+
+ return result
+
+ def test_rustc_unset(self):
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": None })
+ self.assertIn("Environment variable 'RUSTC' is not set.", result.stderr)
+ self.assertIn("This script is intended to be called from Kbuild.", result.stderr)
+
+ def test_bindgen_unset(self):
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": None })
+ self.assertIn("Environment variable 'BINDGEN' is not set.", result.stderr)
+ self.assertIn("This script is intended to be called from Kbuild.", result.stderr)
+
+ def test_cc_unset(self):
+ result = self.run_script(self.Expected.FAILURE, { "CC": None })
+ self.assertIn("Environment variable 'CC' is not set.", result.stderr)
+ self.assertIn("This script is intended to be called from Kbuild.", result.stderr)
+
+ def test_rustc_missing(self):
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": self.missing })
+ self.assertIn(f"Rust compiler '{self.missing}' could not be found.", result.stderr)
+
+ def test_bindgen_missing(self):
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.missing })
+ self.assertIn(f"Rust bindings generator '{self.missing}' could not be found.", result.stderr)
+
+ def test_rustc_nonexecutable(self):
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": self.nonexecutable })
+ self.assertIn(f"Running '{self.nonexecutable}' to check the Rust compiler version failed with", result.stderr)
+
+ def test_rustc_unexpected_binary(self):
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": self.unexpected_binary })
+ self.assertIn(f"Running '{self.unexpected_binary}' to check the Rust compiler version did not return", result.stderr)
+
+ def test_rustc_unexpected_name(self):
+ rustc = self.generate_rustc(f"unexpected {self.rustc_default_version} (a8314ef7d 2022-06-27)")
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
+ self.assertIn(f"Running '{rustc}' to check the Rust compiler version did not return", result.stderr)
+
+ def test_rustc_unexpected_version(self):
+ rustc = self.generate_rustc("rustc unexpected (a8314ef7d 2022-06-27)")
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
+ self.assertIn(f"Running '{rustc}' to check the Rust compiler version did not return", result.stderr)
+
+ def test_rustc_no_minor(self):
+ rustc = self.generate_rustc(f"rustc {'.'.join(self.rustc_default_version.split('.')[:2])} (a8314ef7d 2022-06-27)")
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
+ self.assertIn(f"Running '{rustc}' to check the Rust compiler version did not return", result.stderr)
+
+ def test_rustc_old_version(self):
+ rustc = self.generate_rustc("rustc 1.60.0 (a8314ef7d 2022-06-27)")
+ result = self.run_script(self.Expected.FAILURE, { "RUSTC": rustc })
+ self.assertIn(f"Rust compiler '{rustc}' is too old.", result.stderr)
+
+ def test_rustc_new_version(self):
+ rustc = self.generate_rustc("rustc 1.999.0 (a8314ef7d 2099-06-27)")
+ result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "RUSTC": rustc })
+ self.assertIn(f"Rust compiler '{rustc}' is too new. This may or may not work.", result.stderr)
+
+ def test_bindgen_nonexecutable(self):
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.nonexecutable })
+ self.assertIn(f"Running '{self.nonexecutable}' to check the Rust bindings generator version failed with", result.stderr)
+
+ def test_bindgen_unexpected_binary(self):
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": self.unexpected_binary })
+ self.assertIn(f"Running '{self.unexpected_binary}' to check the bindings generator version did not return", result.stderr)
+
+ def test_bindgen_unexpected_name(self):
+ bindgen = self.generate_bindgen_version(f"unexpected {self.bindgen_default_version}")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"Running '{bindgen}' to check the bindings generator version did not return", result.stderr)
+
+ def test_bindgen_unexpected_version(self):
+ bindgen = self.generate_bindgen_version("bindgen unexpected")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"Running '{bindgen}' to check the bindings generator version did not return", result.stderr)
+
+ def test_bindgen_no_minor(self):
+ bindgen = self.generate_bindgen_version(f"bindgen {'.'.join(self.bindgen_default_version.split('.')[:2])}")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"Running '{bindgen}' to check the bindings generator version did not return", result.stderr)
+
+ def test_bindgen_old_version(self):
+ bindgen = self.generate_bindgen_version("bindgen 0.50.0")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"Rust bindings generator '{bindgen}' is too old.", result.stderr)
+
+ def test_bindgen_new_version(self):
+ bindgen = self.generate_bindgen_version("bindgen 0.999.0")
+ result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })
+ self.assertIn(f"Rust bindings generator '{bindgen}' is too new. This may or may not work.", result.stderr)
+
+ def test_bindgen_libclang_failure(self):
+ for env in (
+ { "LLVM_CONFIG_PATH": self.missing },
+ { "LIBCLANG_PATH": self.missing },
+ { "CLANG_PATH": self.missing },
+ ):
+ with self.subTest(env=env):
+ result = self.run_script(self.Expected.FAILURE, env | { "PATH": os.environ["PATH"], "BINDGEN": "bindgen" })
+ self.assertIn("Running 'bindgen' to check the libclang version (used by the Rust", result.stderr)
+ self.assertIn("bindings generator) failed with code ", result.stderr)
+
+ def test_bindgen_libclang_unexpected_version(self):
+ bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version unexpected [-W#pragma-messages], err: false")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"Running '{bindgen}' to check the libclang version (used by the Rust", result.stderr)
+ self.assertIn("bindings generator) did not return an expected output. See output", result.stderr)
+
+ def test_bindgen_libclang_old_version(self):
+ bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version 10.0.0 [-W#pragma-messages], err: false")
+ result = self.run_script(self.Expected.FAILURE, { "BINDGEN": bindgen })
+ self.assertIn(f"libclang (used by the Rust bindings generator '{bindgen}') is too old.", result.stderr)
+
+ def test_clang_matches_bindgen_libclang_different_bindgen(self):
+ bindgen = self.generate_bindgen_libclang("scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version 999.0.0 [-W#pragma-messages], err: false")
+ result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "BINDGEN": bindgen })
+ self.assertIn("version does not match Clang's. This may be a problem.", result.stderr)
+
+ def test_clang_matches_bindgen_libclang_different_clang(self):
+ cc = self.generate_clang("clang version 999.0.0")
+ result = self.run_script(self.Expected.SUCCESS_WITH_WARNINGS, { "CC": cc })
+ self.assertIn("version does not match Clang's. This may be a problem.", result.stderr)
+
+ def test_rustc_src_core_krustflags(self):
+ result = self.run_script(self.Expected.FAILURE, { "PATH": os.environ["PATH"], "RUSTC": "rustc", "KRUSTFLAGS": f"--sysroot={self.missing}" })
+ self.assertIn("Source code for the 'core' standard library could not be found", result.stderr)
+
+ def test_rustc_src_core_rustlibsrc(self):
+ result = self.run_script(self.Expected.FAILURE, { "RUST_LIB_SRC": self.missing })
+ self.assertIn("Source code for the 'core' standard library could not be found", result.stderr)
+
+ def test_success_cc_unknown(self):
+ result = self.run_script(self.Expected.SUCCESS_WITH_EXTRA_OUTPUT, { "CC": self.missing })
+ self.assertIn("unknown C compiler", result.stderr)
+
+ def test_success_cc_multiple_arguments_ccache(self):
+ clang = self.generate_clang(f"""Ubuntu clang version {self.llvm_default_version}-1ubuntu1
+Target: x86_64-pc-linux-gnu
+Thread model: posix
+InstalledDir: /usr/bin
+""")
+ result = self.run_script(self.Expected.SUCCESS, { "CC": f"{clang} clang" })
+
+ def test_success_rustc_version(self):
+ for rustc_stdout in (
+ f"rustc {self.rustc_default_version} (a8314ef7d 2022-06-27)",
+ f"rustc {self.rustc_default_version}-dev (a8314ef7d 2022-06-27)",
+ f"rustc {self.rustc_default_version}-1.60.0 (a8314ef7d 2022-06-27)",
+ ):
+ with self.subTest(rustc_stdout=rustc_stdout):
+ rustc = self.generate_rustc(rustc_stdout)
+ result = self.run_script(self.Expected.SUCCESS, { "RUSTC": rustc })
+
+ def test_success_bindgen_version(self):
+ for bindgen_stdout in (
+ f"bindgen {self.bindgen_default_version}",
+ f"bindgen {self.bindgen_default_version}-dev",
+ f"bindgen {self.bindgen_default_version}-0.999.0",
+ ):
+ with self.subTest(bindgen_stdout=bindgen_stdout):
+ bindgen = self.generate_bindgen_version(bindgen_stdout)
+ result = self.run_script(self.Expected.SUCCESS, { "BINDGEN": bindgen })
+
+ def test_success_bindgen_libclang(self):
+ for stderr in (
+ f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {self.llvm_default_version} (https://github.com/llvm/llvm-project.git 4a2c05b05ed07f1f620e94f6524a8b4b2760a0b1) [-W#pragma-messages], err: false",
+ f"/home/jd/Documents/dev/kernel-module-flake/linux-6.1/outputs/dev/lib/modules/6.1.0-development/source/scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {self.llvm_default_version} [-W#pragma-messages], err: false",
+ f"scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {self.llvm_default_version} (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false",
+ f"""
+/nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1-p16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (c)
+scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {self.llvm_default_version} [-W#pragma-messages], err: false
+""",
+ f"""
+/nix/store/dsd5gz46hdbdk2rfdimqddhq6m8m8fqs-bash-5.1.0-p16/bin/bash: warning: setlocale: LC_ALL: cannot change locale (c)
+/home/jd/Documents/dev/kernel-module-flake/linux-6.1/outputs/dev/lib/modules/6.1.0-development/source/scripts/rust_is_available_bindgen_libclang.h:2:9: warning: clang version {self.llvm_default_version} (Fedora 13.0.0-3.fc35) [-W#pragma-messages], err: false
+"""
+ ):
+ with self.subTest(stderr=stderr):
+ bindgen = self.generate_bindgen_libclang(stderr)
+ result = self.run_script(self.Expected.SUCCESS, { "BINDGEN": bindgen })
+
+ def test_success_clang_version(self):
+ for clang_stdout in (
+ f"clang version {self.llvm_default_version} (https://github.com/llvm/llvm-project.git 4a2c05b05ed07f1f620e94f6524a8b4b2760a0b1)",
+ f"clang version {self.llvm_default_version}-dev",
+ f"clang version {self.llvm_default_version}-2~ubuntu20.04.1",
+ f"Ubuntu clang version {self.llvm_default_version}-2~ubuntu20.04.1",
+ ):
+ with self.subTest(clang_stdout=clang_stdout):
+ clang = self.generate_clang(clang_stdout)
+ result = self.run_script(self.Expected.SUCCESS, { "CC": clang })
+
+ def test_success_real_programs(self):
+ for cc in ["gcc", "clang"]:
+ with self.subTest(cc=cc):
+ result = self.run_script(self.Expected.SUCCESS, {
+ "PATH": os.environ["PATH"],
+ "RUSTC": "rustc",
+ "BINDGEN": "bindgen",
+ "CC": cc,
+ })
+
+if __name__ == "__main__":
+ unittest.main()