Darwin build environment

A Darwin installation is useful for building MacOS X binaries without using MacOS X itself.

Darwin 8.0.1 (corresponding to MacOS X 10.4) seems to be the latest available full installer image, at least, until the PureDarwin project produces a new one. The steps to install it are as follows:

1. Download and extract installation CD image:
   • Website: http://developer.apple.com/Darwin/
   • ISO file: http://www.opensource.apple.com/darwinsource/images/darwinx86-801.iso.gz (under “Darwin 8.0.1 Installer CD”)
     • MD5 (darwinx86-801.iso.gz) = dbd260dda994093a11c31afbe624aa34
     • SHA256 (darwinx86-801.iso.gz) = a26f9f14795fe0687bd647d41b712464c9a61430736fae4aa0fdb89207e1a62a
   • $ gunzip darwinx86-801.iso.gz
2. Set up QEMU VM:
   • $ qemu-img create -f qcow2 darwin.qcow2 10G
3. Start VM:
   • $ qemu -net none -drive file=darwin.qcow2,cache=writeback -cdrom darwinx86-801.iso -boot d
   • Other useful qemu options: -m 512 and -enable-kqemu
4. Installation steps:
   • Select disk
   • ”2” (manual partitioning)
   • ”y” (initialize MBR)
   • In fdisk:
     • > auto hfs
     • Ignore the warning
     • > update
     • > write
     • > quit
   • Enter partition name
   • ”yes” (clean install)
   • ”darwin” (volume name)
   • Continue (reboot)
   • Select disk
   • ”3” (existing partition)
   • Enter partition name
   • ”hfs” (HFS+ filesystem)
   • ”yes” (clean install)
   • ”darwin” (volume name)
   • Wait for installation to complete
   • Enter root password
   • ”darwin” (hostname)
   • ”3” (spawn shell)
   • # halt

1. Start VM:
   • $ qemu -net none -drive file=darwin.qcow2,cache=writeback
2. Log in as root
3. Some configuration:
   • # echo "alias ll='ls -l'" >>/etc/bashrc
   • # echo "alias la='ls -lA'" >>/etc/bashrc
4. # halt

1. To enable networking use -net nic,model=rtl8139 -net user
2. The VM fails to boot sometimes with “panic(…): nfs_boot_init failed”; if this happens, restart it.

The GCC versions provided with 8.0.1 are quite dated. The steps to upgrade to a later version using source packages are as follows:

1. Make a temporary filesystem for building (this is optional, but helps to keep the main disk image file smaller):
   • $ qemu-img create -f qcow2 scratch.qcow2 10G
   • Start VM:
     • $ qemu -net none -drive file=darwin.qcow2,cache=writeback -drive file=scratch.qcow2,cache=writeback
   • Log in as root
   • Figure out which disk number corresponds to the scratch drive:
     • # mount
     • # ls /dev/rdisk*
     • It's the one that isn't mounted and doesn't have any partitions (sn-suffixed files)
     • It's probably /dev/rdisk1
   • # newfs_hfs -v scratch /dev/rdisk1
   • # halt
2. Download source packages:
   • Website: http://opensource.apple.com
   • dyld: http://www.opensource.apple.com/tarballs/dyld/dyld-46.16.tar.gz (under “Mac OS X 10.4.11.x86”)
     • MD5 (dyld-46.16.tar.gz) = a97d4d2a77b3e5f2ff9757007cae0596
     • SHA256 (dyld-46.16.tar.gz) = cf753c3a2cef239b1334c56603b36d206f54d613daa078ce0f2d73c3e2a0dd82
   • cctools: http://opensource.apple.com/tarballs/cctools/cctools-667.3.tar.gz (under “Mac OS X 10.5.8”)
     • MD5 (cctools-667.3.tar.gz) = 518166e6ddbd0287bebee3d00fddee7d
     • SHA256 (cctools-667.3.tar.gz) = 1ba43a3d3bcf138a196e714666927c706857226adccf2e69db4f08e940631c0d
   • gcc: http://opensource.apple.com/tarballs/gcc_42/gcc_42-5531.tar.gz (under “Mac OS X 10.5.8”)
     • MD5 (gcc_42-5531.tar.gz) = 959ecd224cea2477d57b1c3026fe05ee
     • SHA256 (gcc_42-5531.tar.gz) = 924862e2092fa11754ec9226a0de4772d831c40500231728055d00d1bb3a632e
   • Place source packages one directory, e.g. packages
   • Also place Makefile.dyld listed at the bottom of this page in that directory. Darwin doesn't include Xcode, which is required to build some packages, including dyld. This Makefile provides alternative build instructions.
3. Start VM:
   • $ qemu -net none -drive file=darwin.qcow2,cache=writeback -drive file=scratch.qcow2,cache=writeback -drive file=fat:rw:packages
4. Log in as root
5. Upgrade dyld:
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/dyld-46.16.tar.gz
   • # cp /Volumes/QEMU\ VVFAT/Makefile.dyld dyld-46.16/Makefile
   • # sed -i.tmp '/case CPU_TYPE_X86_64:/,/break;/d;/__pthread_tsd_first/d' dyld-46.16/src/dyld.cpp
   • # ln -s libstdc++.a /usr/lib/gcc/i686-apple-darwin8/4.0.0/libstdc++-static.a
   • # bsdmake -C dyld-46.16 VERSION_DYLD=46 install
6. Upgrade cctools:
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/cctools-667.3.tar.gz
   • # sed -i.tmp '/install.*strip/{h;s/strip/seg_hack/gp;g;s/ -s / /;}' cctools-667.3/misc/Makefile
   • # sed -i.tmp s/ld_classic/ld/ cctools-667.3/ld/Makefile
   • # make -C cctools-667.3 install
   • Note: the new assembler will often show the warning “indirect jmp without '*'” on code generated by the old GCC.
7. Upgrade gcc:
   • # ln -s true /usr/bin/dsymutil
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/gcc_42-5531.tar.gz
   • # mkdir gcc-build; cd gcc-build
   • # ../gcc_42-5531/configure --prefix=/usr --program-suffix=-4.2 --disable-nls
   • # gnumake
   • # gnumake install
   • Note: GCC is now installed as gcc-4.2

1. Upgrade dyld to an earlier version:
   • This version compiles without modification, but gcc_42-5531 doesn't work with it.
   • Source package: http://opensource.apple.com/tarballs/dyld/dyld-44.2.tar.gz (under “Mac OS X 10.4.3”)
     • MD5 (dyld-44.2.tar.gz) = b681e6f48a3100aaa8409e1f69ffe90d
     • SHA256 (dyld-44.2.tar.gz) = 59854fa487374a746a55066d8fe1de95b13c0d4f81c65958dee6ea1d18b65484
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/dyld-44.2.tar.gz
   • # cp /Volumes/QEMU\ VVFAT/dyld/Makefile.dyld dyld-44.2/Makefile
   • # bsdmake -C dyld-44.2 install
2. Upgrade the kernel (xnu):
   • This is necessary to upgrade libc, but breaks support for FAT (and thus the QEMU virtual FAT drive). Support for FAT is in the msdosfs package which requires Xcode to build.
   • An alternative to the virtual FAT drive for transferring files is using tar directly with an unpartitioned drive.
   • Source package: http://www.opensource.apple.com/tarballs/xnu/xnu-792.6.22.tar.gz (under “Mac OS X 10.4.3”)
     • MD5 (xnu-792.6.22.tar.gz) = bb7d4cd3edf7bd5bac7603c646d9a417
     • SHA256 (xnu-792.6.22.tar.gz) = 7ef985e1b2d3c2a77920fdbf5e4eddbe9e327fc2ddc150b1463c719e8de72bfb
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/xnu-792.6.22tar.gz
   • # make -C xnu-792.6.22 DSTROOT= install
3. Upgrade libc:
   • Source package: http://opensource.apple.com/tarballs/Libc/Libc-391.2.3.tar.gz (under “Mac OS X 10.4.3”)
     • MD5 (Libc-391.2.3.tar.gz) = ff06f0b37c19e602ff6b4d08b1e7f746
     • SHA256 (Libc-391.2.3.tar.gz) = a8bac3a29ea6df05fe48a856680a04deef28e21887587acec59c1f58c7593f19
   • # cd /Volumes/scratch
   • # tar xzf /Volumes/QEMU\ VVFAT/Libc-391.2.3.tar.gz
   • # sed -i.tmp s/NOTIFY_STATUS_OK/0/g Libc-391.2.3/gen/{asl.c,syslog.c}
   • # mkdir Libc-391.2.3/OBJROOT
   • # make -C Libc-391.2.3 OBJROOT=`pwd`/Libc-391.2.3/OBJROOT install

Makefile.dyld

VERSION_DYLD?=44
 
ENV=MACOSX_DEPLOYMENT_TARGET=10.4
CC=gcc-4.0
CXX=g++-4.0
 
all: dyld libdyldapis.a
install: install_libdyld install_dyld
 
# dyld
 
CFLAGS_DYLD=-Os -I./include -Wmost -Wno-four-char-constants -Wno-unknown-pragmas -mdynamic-no-pic
CXXFLAGS_DYLD=-fno-rtti
.if ${VERSION_DYLD} == 44
    LDFLAGS_DYLD=/usr/lib/gcc/i686-apple-darwin8/4.0.0/libstdc++.a
.else
    LDFLAGS_DYLD=-lstdc++-static
.endif
LDFLAGS_DYLD+= -seg1addr 8fe00000 -exported_symbols_list src/dyld.exp -nostdlib /usr/local/lib/system/libc.a -lgcc_eh -lgcc -Wl,-e,__dyld_start -Wl,-dylinker -Wl,-dylinker_install_name,/usr/lib/dyld
 
OBJ_DYLD=
.for i in dyldStartup.s stub_binding_helper.s
OBJ_DYLD+= ${i:R}.o
${i:R}.o: src/${i}
	${ENV} ${CC} ${CFLAGS_DYLD} -c $< -o $@
.endfor
 
.for i in dyldExceptions.c glue.c dyld_vers.c
OBJ_DYLD+= ${i:R}.o
${i:R}.o: src/${i}
	${ENV} ${CC} ${CFLAGS_DYLD} -c $< -o $@
.endfor
 
.for i in dyld_gdb.cpp dyld.cpp dyldAPIs.cpp dyldInitialization.cpp dyldNew.cpp ImageLoader.cpp ImageLoaderMachO.cpp
OBJ_DYLD+= ${i:R}.o
${i:R}.o: src/${i}
	${ENV} ${CXX} ${CFLAGS_DYLD} ${CXXFLAGS_DYLD} -c $< -o $@
.endfor
 
src/dyld_vers.c:
	echo 'const unsigned char dyldVersionString[] __attribute__ ((used)) = "@(#)PROGRAM:dyld PROJECT:dyld-${VERSION_DYLD} DEVELOPER:root BUILT" __DATE__ " " __TIME__ "\n";' > $@
	echo 'const double dyldVersionNumber __attribute__ ((used)) = (double)${VERSION_DYLD};' >> $@
 
dyld: ${OBJ_DYLD}
	${ENV} ${CXX} ${CFLAGS_DYLD} -o $@ $> ${LDFLAGS_DYLD} 
	strip -S $@
 
install_dyld: dyld
	install dyld /usr/lib/
 
# libdyld
 
CFLAGS_LIBDYLD=-Os -I./include -Wmost -Wno-four-char-constants -Wno-unknown-pragmas
CXXFLAGS_LIBDYLD=-fno-exceptions -fno-rtti
 
HDR_LIBDYLD=src/dyldLibSystemThreadHelpers.h include/mach-o/dyld_debug.h
OBJ_LIBDYLD=
.for i in dyld_debug.c
OBJ_LIBDYLD+= ${i:R}.o
${i:R}.o: src/${i} ${HDR_LIBDYLD}
	${ENV} ${CC} ${CFLAGS_LIBDYLD} -c $< -o $@
.endfor
 
.for i in dyldLock.cpp dyldAPIsInLibSystem.cpp 
OBJ_LIBDYLD+= ${i:R}.o
${i:R}.o: src/${i} ${HDR_LIBDYLD}
	${ENV} ${CXX} ${CFLAGS_LIBDYLD} ${CXXFLAGS_LIBDYLD} -c $< -o $@
.endfor
 
libdyldapis.a: ${OBJ_LIBDYLD}
	ar rcs $@ $>
 
install_libdyld: libdyldapis.a
	install -m 644 libdyldapis.a /usr/local/lib/system/
	install -m 644 include/dlfcn.h /usr/include/
	install -m 644 include/mach-o/dyld_debug.h include/mach-o/dyld.h /usr/include/mach-o/
	install -m 644 include/mach-o/dyld_gdb.h include/mach-o/dyld_priv.h /usr/local/include/mach-o/
	rm -f /usr/local/lib/system/libdyldapis_profile.a /usr/local/lib/system/libdyldapis_debug.a
	ln -s libdyldapis.a /usr/local/lib/system/libdyldapis_profile.a
	ln -s libdyldapis.a /usr/local/lib/system/libdyldapis_debug.a