I searched and didn't find, and had to figure stuff out... I hope to save you the trouble.

Wednesday, April 22, 2009

Build squashfs tools for Mac OS X

To squash/unsquash a Linux squashfs filesystem on a Mac, you can build the squashfs tools.

A few modifications of the source are required for OS X:
  • The FNM_EXTMATCH flag doesn't exist in the fnmatch library on OS X. This just means that you can't use glob patterns with mksquashfs.
  • The constants to figure out how many CPUs the machine has are in the sysctl header instead of the sysinfo header.
  • The C99 compiler has different semantics for the inline keyword, so inline functions should also be static.
  • Instead of llistxattr(), lgetxattr(), lsetxattr(), Apple uses a XATTR_NOFOLLOW flag on the non-link equivalent functions.

Here's a complete download-and-build recipe:
curl -OL http://iweb.dl.sourceforge.net/project/squashfs/squashfs/squashfs4.2/squashfs4.2.tar.gz
tar xzvf squashfs4.2.tar.gz
cd squashfs4.2/squashfs-tools

sed -i.orig 's/FNM_EXTMATCH/0/; s/sysinfo.h/sysctl.h/; s/^inline/static inline/' mksquashfs.c unsquashfs.c

cat <<END >> xattr.h
#define llistxattr(path, list, size) \
  (listxattr(path, list, size, XATTR_NOFOLLOW))
#define lgetxattr(path, name, value, size) \
  (getxattr(path, name, value, size, 0, XATTR_NOFOLLOW))
#define lsetxattr(path, name, value, size, flags) \
  (setxattr(path, name, value, size, 0, flags | XATTR_NOFOLLOW))
END

make
sudo cp mksquashfs unsquashfs /usr/local/bin

Usage notes:

The filesystem you're unsquashing probably has root-owned files in it, so you'll have to use sudo to run unsquashfs as root.

The limit of open files is somewhat low on a Mac, so you'll get a bunch of errors if you try to unsquash a whole filesystem:
write_file: failed to create file squashfs-root/path/to/file, because Too many open files
Just change the limit in the shell before running unsquashfs:
ulimit -n 2000
sudo unsquashfs filesystem.squashfs

There seems to be a bug when drawing the progress bar that causes Floating point exception, which you can workaround by using the -n flag.

11 comments:

  1. For completeness' sake, I'll point out that the syntax of the first sed command only works in bash (not tcsh).

    Also, the cp command needs to be prefixed with sudo.

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Great!!! Just what I need to work with Android Systems in sqsh format.
    Updated location: http://sourceforge.net/projects/squashfs/files/

    I'm missing the "make", where can I find to install make?

    ReplyDelete
  5. Thanks, Benjamin. I've updated the curl command based on the URL.
    The make utility is installed as part of XCode from your OS X install DVD.

    ReplyDelete
  6. Great! Thanks. I'm able to unsquashfs now.

    I had to do the following:

    - The /usr/local/bin directory had to be created first.
    - su to root before copying to /usr/local/bin

    Thanks Again!

    ReplyDelete
  7. hmm 4.2 won't build on mac with this formula, output:

    make
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o mksquashfs.o mksquashfs.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o read_fs.o read_fs.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o sort.o sort.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o swap.o swap.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o pseudo.o pseudo.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o compressor.o compressor.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o gzip_wrapper.o gzip_wrapper.c
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o xattr.o xattr.c
    xattr.c: In function ‘read_xattrs_from_system’:
    xattr.c:133: warning: implicit declaration of function ‘llistxattr’
    xattr.c:185: warning: implicit declaration of function ‘lgetxattr’
    cc -O2 -I. -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -DGZIP_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o read_xattrs.o read_xattrs.c
    cc mksquashfs.o read_fs.o sort.o swap.o pseudo.o compressor.o gzip_wrapper.o xattr.o read_xattrs.o -lpthread -lm -lz -o mksquashfs
    Undefined symbols:
    "_lgetxattr", referenced from:
    _read_xattrs in xattr.o
    _read_xattrs in xattr.o
    "_llistxattr", referenced from:
    _read_xattrs in xattr.o
    _read_xattrs in xattr.o
    ld: symbol(s) not found
    collect2: ld returned 1 exit status
    make: *** [mksquashfs] Error 1

    ReplyDelete
    Replies
    1. I've updated the modifications needed and the recipe for squashfs 4.2

      Delete
  8. It seems something broke after Apple changed something.

    Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

    is giving

    make
    cc mksquashfs.o read_fs.o sort.o swap.o pseudo.o compressor.o gzip_wrapper.o xattr.o read_xattrs.o -lpthread -lm -lz -o mksquashfs
    Undefined symbols for architecture x86_64:
    "_add_dir_entry", referenced from:
    _scan1_encomp_readdir in mksquashfs.o
    _scan1_single_readdir in mksquashfs.o
    _dir_scan1 in mksquashfs.o
    _dir_scan2 in mksquashfs.o
    ld: symbol(s) not found for architecture x86_64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    make: *** [mksquashfs] Error 1

    ReplyDelete
    Replies
    1. Thanks for calling out the error, Rubens.

      The current XCode compiler adheres to C99, which has a different interpretation of inline than GCC did.
      http://clang.llvm.org/compatibility.html#inline

      I've updated the first sed command to add 'static' to all the inline functions, and with that it will build again.

      Delete
  9. The url to download the tar file from no longer works. Use this instead:

    curl -OL http://iweb.dl.sourceforge.net/project/squashfs/squashfs/squashfs4.2/squashfs4.2.tar.gz

    Note these instructions don't work for squashfs4.3. I get this error when running make:

    $make
    cc -g -O2 -I. -I./LZMA/lzma465/C -I./LZMA/lzmalt -I./LZMA/lzmadaptive/C/7zip/Compress/LZMA_Lib -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_GNU_SOURCE -DCOMP_DEFAULT=\"gzip\" -Wall -Werror -DGZIP_SUPPORT -DLZMA_SUPPORT -DXZ_SUPPORT -DLZO_SUPPORT -DXATTR_SUPPORT -DXATTR_DEFAULT -c -o unsquashfs.o unsquashfs.c
    unsquashfs.c:36:10: fatal error: 'sys/sysmacros.h' file not found
    #include
    ^~~~~~~~~~~~~~~~~
    1 error generated.
    make: *** [unsquashfs.o] Error 1

    ReplyDelete

Followers