elisoli' blog

Flutter on musl libc

This is an attempt of developing projects with Flutter on a musl based system. Although Flutter doesn't support musl directly we can replace glibc-dependent binaries by static ones.

This guide is not definitive and might stop working on later versions of Flutter.

Overview

If you have ever tried running Flutter on a musl based system you might have faced this error:

/home/user/.flutter/bin/internal/shared.sh: line 273: /home/user/.flutter/bin/cache/dart-sdk/bin/dart: cannot execute: required file not found

Or if you have gcompat install it simply hangs indefinitly. This happens because the dart binary that's shipped with the dart-sdk downloaded by Flutter is linked agaisnt glibc. Our first attempt will be to compile dart from source and replace this binary.

You might think about replacing this binary by the one at flutter/bin/dart but this won't work, instead it will generate another weird error.

Building dart from source

As expected building dart form source requires some additional tools as gn and depot_tools. Let alone small patches in files and binaries that are shipped with the source.

For that I made a small script to make the build easier to manage. You'll need these dependencies installed:

#!/bin/sh -e

# Use pre-existing folder

cleanup() {
	rm -rf "$DIR"
}

[ $# -eq 0 ] && {
	DIR=$(mktemp -d)
	cd $DIR
	trap cleanup EXIT INT HUP

	echo $DIR
}

# Build gn

build_gn() {
	git clone https://gn.googlesource.com/gn
	cd gn
	python build/gen.py
	ninja -C out gn
	cd ..
}

[ ! -d 'gn' ] && build_gn

# Fetch depot tools

fetch_depot_tools() {
	git clone --depth 1 https://chromium.googlesource.com/chromium/tools/depot_tools.git
}

[ ! -d 'depot_tools' ] && fetch_depot_tools

# Fetch dart source

fetch_dart() {
	export DEPOT_TOOLS_UPDATE=0
	export DEPOT_TOOLS_BOOTSTRAP_PYTHON3=0
	export VPYTHON_BYPASS="manually managed python not supported by chrome operations"

	depot_tools/fetch dart

	cd sdk
	git fetch --tags origin
	git checkout 04bcd1036cdc799ac6564988f159ee454d42c822
	cd ..

	depot_tools/gclient sync -D -r sdk@04bcd1036cdc799ac6564988f159ee454d42c822
}

[ ! -d 'sdk' ] && fetch_dart

# Replace binaries for native ones

rm -f sdk/buildtools/gn
cp gn/out/gn sdk/buildtools/gn

rm -rf sdk/buildtools/linux-x64/clang
mkdir -p sdk/buildtools/linux-x64/clang/bin

# Wrappers for the compilers: drop glibc target/sysroot, disable -Werror
for t in clang clang++; do
	real="$(command -v $t)" || continue
	cat > "sdk/buildtools/linux-x64/clang/bin/$t" <<EOF
#!/bin/sh
n=\$#
i=0
while [ \$i -lt \$n ]; do
	a="\$1"; shift
	case "\$a" in
		--target=*|--sysroot=*|-Wl,--icf=*) ;;
		*) set -- "\$@" "\$a" ;;
	esac
	i=\$((i+1))
done
exec $real "\$@" -Wno-error
EOF
	chmod +x "sdk/buildtools/linux-x64/clang/bin/$t"
done

# Plain symlinks for the rest
for t in clang-cpp llvm-ar llvm-nm llvm-objcopy llvm-strip llvm-readelf llvm-objdump llvm-ranlib; do
	ln -sf "$(command -v $t)" sdk/buildtools/linux-x64/clang/bin/$t
done

# Create the stamp for clang
mkdir -p sdk/buildtools/linux-x64/clang/.versions
cat > sdk/buildtools/linux-x64/clang/.versions/clang.cipd_version <<'EOF'
{
  "package_name": "local/system-clang",
  "instance_id": "system-clang-musl"
}
EOF

# Sysroot: remove the Debian one first, then recreate with the stamp
rm -rf sdk/buildtools/sysroot/linux
mkdir -p sdk/buildtools/sysroot/linux/.versions

for d in usr lib lib64; do
	[ -e /$d ] || continue
	ln -s /$d sdk/buildtools/sysroot/linux/$d
done

cat > sdk/buildtools/sysroot/linux/.versions/sysroot.cipd_version <<'EOF'
{
  "package_name": "local/system-sysroot",
  "instance_id": "system-sysroot-musl"
}
EOF

# Build dart

cd sdk
./tools/build.py --mode release runtime

# Configure networking

[ ! -f '/etc/ssl/certs/ca-certificates.crt' ] && {
	mkdir /etc/ssl/certs
	ln -s /etc/ssl/cert.pem /etc/ssl/certs/ca-certificates.crt
}

After finishing compiling you can replace flutter's dart-sdk's binary by the one just compiled.

cp sdk/out/ReleaseX64/dart ~/.flutter/bin/cache/dart-sdk/bin/dart

Replacing aapt2 binary

If you try to build a project now you still get an error saying aapt2 couldn't start the daemon.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:mergeReleaseResources'.
> Multiple task action failures occurred:
   > A failure occurred while executing com.android.build.gradle.internal.res.Aapt2CompileRunnable
      > AAPT2 aapt2-9.1.0-14792394-linux Daemon #1: Daemon startup failed
        This should not happen under normal circumstances, please file an issue if it does.
   > A failure occurred while executing com.android.build.gradle.internal.res.Aapt2CompileRunnable
      > AAPT2 aapt2-9.1.0-14792394-linux Daemon #3: Daemon startup failed
        This should not happen under normal circumstances, please file an issue if it does.
   > A failure occurred while executing com.android.build.gradle.internal.res.Aapt2CompileRunnable
      > AAPT2 aapt2-9.1.0-14792394-linux Daemon #2: Daemon startup failed
        This should not happen under normal circumstances, please file an issue if it does.
   > A failure occurred while executing com.android.build.gradle.internal.res.Aapt2CompileRunnable
      > AAPT2 aapt2-9.1.0-14792394-linux Daemon #4: Daemon startup failed
        This should not happen under normal circumstances, please file an issue if it does.
   > A failure occurred while executing com.android.build.gradle.internal.res.Aapt2CompileRunnable
      > AAPT2 aapt2-9.1.0-14792394-linux Daemon #0: Daemon startup failed
        This should not happen under normal circumstances, please file an issue if it does.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights from a Build Scan (powered by Develocity).
> Get more help at https://help.gradle.org.

BUILD FAILED in 6s
Running Gradle task 'assembleRelease'...                            7.1s
Gradle task assembleRelease failed with exit code 1

That happens because the aapt2 binary Flutter is executing is glibc-linked.

~ $ find .gradle/ -name 'aapt2-9.1.0-14792394-linux'
.gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux
~ $ ldd .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/
aapt2
/lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x7fe247986000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fe247eb4000)
ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2 (0x7fe24797f000)
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __memcpy_chk: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __isnanf: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __fprintf_chk: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __longjmp_chk: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __vsnprintf_chk: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: strtoll_l: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: strtoull_l: symbol not found
Error relocating .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2: __longjmp_chk: symbol not found

To solve that we can just copy the aapt2 binary distributed with android-sdk that is static.

cp ~/.android/android-sdk/build-tools/36.1.0/aapt2 .gradle/caches/9.3.1/transforms/35fc7c9aabf3963476dbd7a99d18fea8/transformed/aapt2-9.1.0-14792394-linux/aapt2

References

Read more