The optimised version of 7-Zip can't be built from source

If I was going to lean more into clickbait, I’d title this “7-Zip isn’t open source”. It’s likely if you’re on Linux you’re actually using a fork of it called p7zip, rather than the upstream project. If you’re on Fedora 40, you’re using a years-old version ¹.

The 7-Zip author also doesn’t believe in (public?) source control, dumping quarterly archives of source code online, but I’ll leave that particular gripe there.

I thought I’d try to build 7-Zip from source to be able to use a newer version on my systems. How hard could it be?

#8 [5/6] WORKDIR /src/7z/CPP/7zip/Bundles/Alone2
#8 DONE 0.1s

#9 [6/6] RUN make -j -f ../../cmpl_gcc_x64.mak
#9 0.307 mkdir -p b/g_x64
#9 0.317 asmc -nologo -elf64 -DABI_LINUX -Fob/g_x64/ ../../../../Asm/x86/LzmaDecOpt.asm
#9 0.317 make: asmc: No such file or directory
#9 0.317 make: *** [../../7zip_gcc.mak:1310: b/g_x64/LzmaDecOpt.o] Error 127

That’s weird, what’s asmc? It turns out it’s a “nasm compatible assembler”. Firstly, it looks like 7-Zip started out as a Windows-only project and secondly the “optimised build” of 7-Zip requires some hand rolled (?) assembly routines to be included. I’m not going to argue, I can’t read assembly and I don’t want my build of 7-Zip to be slow.

Following the dependency rabbit hole, I try to build https://github.com/nidud/asmc from source.

At this point I’m using an Alpine container and I get the following error:

 > [ 6/10] RUN make && make install:
0.249 chmod a+x ../../bin/asmc64
0.250 ../../bin/asmc64 -fpic -Zp8 -Cs -Iinc -I../../include ../libc/fltintrn/*.asm src/*.asm
0.251 /bin/sh: ../../bin/asmc64: not found

Huh? Why is building asmc64 trying to use asmc64? As the author says ² it’s a chicken and egg problem. This can also be seen as a bootstrapping problem ³. Hopefully, you’d have a version of asmc that could be built with gcc or similar, then use that version of asmc to build your final version of asmc.

The author of asmc has instead decided to have a large bin folder included in their source tree, which includes prebuilt versions of asmc for a small variety of platforms. Even if I did want to use the prebuilt version, it’s dynamically linked to glibc so won’t work on alpine.

In the same thread from the author, they mention that asmc is a fork of JWasm ⁴, which was written in C. Maybe it’s as “easy” as swapping out one for the other - using gcc to build jwasm, then using jwasm to build asmc?

jwasm -fpic -Zp8 -Cs -Iinc -I../../include ../libc/fltintrn/*.asm src/*.asm
Warning A4109: Invalid command-line option: -fpic
Warning A4109: Invalid command-line option: -Cs
../../include/fltintrn.inc(1) : Error A2209: Syntax error: .
 ../../include/fltintrn.inc(1): Included by
  ../libc/fltintrn/_addq.asm(6): Main line code
../../include/fltintrn.inc(3) : Error A2209: Syntax error: define
...

Ugh. This is a bit beyond my knowledge to fix. I can probably drop those two unsupported flags out, but what about the syntax errors?

Searching back through the commit history of asmc, I could see references to Open Watcom, but it took me some time to track down a version of the Makefile that didn’t assume you already had asmc built.

Most commits in this repository only have a version number as a commit message - the following will refer to commits as versions.

version 2.34 still has references to watcom, but seems to rely on a pre-existing copy of asmc.

version 2.33 seems to assume you’d build the Linux version using Open Watcom on Windows.

I should probably be doing something more productive, like cleaning my house.

If I come back to this, I can either try to use OpenWatcom v2 on Linux, with a hacked up Makefile, or try to grab a 2021 (?) version of OpenWatcom and run it on WINE

I think the build process will be something like:

1. Build asmc 2.32 for Linux using OpenWatcom in WINE
2. Build asmc 2.33 for Linux using asmc 2.32 (probably needs 32 bit userland)
3. Build asmc 2.34 for Linux
4. Build asmc64 2.34+ using asmc 2.34
5. Hope like hell the current version of asmc64 can be built using asmc64 2.34
6. Finally get back to building 7-Zip

I guess the only real points I can make with this article are: document your builds and build systems and don’t put binary blobs in source control.

Sigh.

Someone smarter than me, please work out the performance impact of not having the assembly routines in 7-Zip - there is a “GCC-only” Makefile and maybe this whole exercise was pointless.