Mac OS X and IOS Internals

Jonathan Levin

Mentioned 8

Presents an architectural overview of Mac OS X and iOS, covering such topics as system startup, processes, security, internal apps, XNU, and device drivers.

More on Amazon.com

Mentioned in questions and answers.

I'm trying to figure out how to get a list of and documentation for the system calls available in the XNU kernel in OSX. I've googled around quite a bit, but haven't been able to find anything of use. As I understand the calling conventions match BSD, is that correct?

Thanks

XNU's system calls are, indeed, following the BSD/POSIX model, but are a subset: There are quite a few additional calls proprietary to Apple - for example, stack_snapshot (#365), iopolicy (322), proc_info(336) and others. Apple keeps those undocumented, but you can find their source in XNU. You can call undocumented system calls explicitly (by using the sys call macro and specifying the call # as first argument), but some (e.g. proc_info) are wrapped by Darwin libraries (e.g. libproc, which is part of libSystem)

syscalls.master generates the /usr/include/sys/syscall.h file, which will also give you the system call names (but not the prototypes). Singh's book describes some sys calls, but is vastly outdated, since Tiger had fewer than the 439 (in iOS6: 443) calls XNU has today. A more recent book can be found in http://www.amazon.com/Mac-OS-iOS-Internals-Apples/dp/1118057651, and its appendix actually documents all system calls (and Mach Traps), in detail.

I'm a bit confused on the best way to compile the latest version of the XNU kernel. I've seen lots of instructions for older kernels that came with Mac OS X 10.4 but the newer sources lack a lot of the things that the instructions contain. Just running make on the XNU kernel source brings a lot of errors about not finding ctfconvert, ctfmerge and ctfdump. Does anyone have a good "howto" to build a new kernel?

A new book by Wiley details the complete set of how-to in chapter 9.

Try this:

#
# Getting C++ filter 
#
$ curl http://opensource.apple.com/tarballs/cxxfilt/cxxfilt-9.tar.gz > cxx.tar.gz
$ tar xvf cxx.tar.gz
$ cd cxxfilt-9
$ mkdir -p build obj sym
$ make install RC_ARCHS="i386 x86_64" RC_CFLAGS="-arch i386 -arch x86_64 -pipe" \
RC_OS=macos RC_RELEASE=Lion SRCROOT=$PWD OBJROOT=$PWD/obj \ SYMROOT=$PWD/sym DSTROOT=$PWD/build
#
# Getting DTrace – This is required for ctfconvert, a kernel build tool 
#
$ curl http://opensource.apple.com/tarballs/dtrace/dtrace-90.tar.gz > dt.tar.gz
$ tar zxvf dt.tar.gz
$ cd dtrace-90
$ mkdir -p obj sym dst
$ xcodebuild install -target ctfconvert -target ctfdump -target ctfmerge \ ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym \ DSTROOT=$PWD/dst
#
# Getting Kext Tools 
#
$ wget http://opensource.apple.com/tarballs/Kext_tools/Kext_tools-180.2.1.tar.gz \ > kt.tar.gz
$ tar xvf kt.tar.gz
$ cd Kext_tools-180.2.1
$ mkdir -p obj sym dst
$ xcodebuild install -target Kextsymboltool -target setsegname \ ARCHS="i386 x86_64" SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym \
DSTROOT=$PWD/dst
#
# Getting Bootstrap commands – newer versions are available, but would # force xcodebuild
#
$ curl http://opensource.apple.com/tarballs/bootstrap_cmds/bootstrap_cmds-72.tar.gz \ > bc.tar.gz
$ tar zxvf bc.tar.gz
$ cd bootstrap_cmds-84
$ mkdir -p obj sym dst
$ make install RC_ARCHS="i386" RC_CFLAGS="-arch i386 -pipe" RC_OS=macos \
RC_RELEASE=Lion SRCROOT=$PWD OBJROOT=$PWD/obj SYMROOT=$PWD/sym DSTROOT=$PWD/dst

The tar ball versions are now different (e.g. DTrace is 96, not 90) but this should work to satisfy dependencies. Once you have them, you just run the usual make (make ARCH_CONFIGS=" X86_64" KERNEL_CONFIGS="RELEASE"). You might want to add DEBUG, to get the great debug and trace messages which are disabled by default.

This works with XCode 4.4. Just tried it now, actually.

There's http://www.amazon.com/gp/product/0321278542/ but it looks a bit dated. Specificaly, it talks about bits that are no longer in use (powerpc bootloader for example) on modern machines.

So, my question really is: is there any other book, as comprehensive and detailed as this one?

EDIT: There's a new book which seems to be relevant: Mac OS X Internals - To The Apple's Core.

No, but it's still a great book. The internal hasn't changed that much; once you get the broad idea by reading that book, you can directly go to http://opensource.apple.com/ and read the latest kernel source code without problems.

And there's also the upcoming http://www.amazon.com/Mac-OS-iOS-Internals-Apples/dp/1118057651 - picking up where Singh's excellent book left off. FYI Yuji, much has indeed changed internally. Mach hasn't, but launchd has been revamped, GCD and XPC is totally new, the porting to ARM is new, and XNU has had about 40-50 new system calls since then.

I've been playing around with/debugging/disassembling binaries on the iPhone.

The first obstacle is that the binaries are encrypted and dissembler can not read them. This can be overcome by dumping the decrypted file content from gdb.

My question is about the decryption of binaries (which is done on the fly upon program start) for all MACH-O executable which have the encryption_id in LC_ENCRYPTION_INFO section, set to 1.

  1. is there a tool that decrypts files for you? Any recommendations?
  2. is there info available on how the process works? Apparently it's AES encryption? What key is used? Is it easy to replicate with your own program?

Thanks for any pointers!

Ethical questions about hacking/cracking/etc aside, let's discuss the details:

  • In OS X the binary decryption is performed by a Don't Steal MacOS X.Kext (affectionately known as DSMOS).
  • In iOS the binary decryption is performed by the FairPlay kext.

    In both cases, the kernel Mach-O loader is responsible for this. If you really need to go into the detail, it's the Apple Protect pager which handles the decryption. In XNU parlance a "pager" is a component which is responsible for taking VM pages and retrieving them from a backing store (swap, memory mapped file, etc). It's the Apple Protect Pager which calls on the kext (one of the above two)

    There's a great book covering the kernel specifics - http://www.amazon.com/Mac-OS-iOS-Internals-Apples/dp/1118057651

    As you correctly stated, this encryption is easily defeated - on a jailbroken device (or a PC with root privileges) you can use the Mach VM APIs to read the decrypted image. That's actually mentioned in the above book, with a sample tool to do that (among other VM tricks). Alternatively, Stefan Esser has a simple tool which decrypts binaries by injecting a simple .dylib into the process address space when loaded, using DYLD_INSERT_LIBRARIES. github.com › stefanesser › dumpdecrypted.

hope this helps

TG

I am developing a iPhone app like " Eject device" on computer. which means, when I connect USB cable for connecting to computer, this application just disconnect that connection by a single Click. I was googled about this, and I got some idea about EAAccessoryManager . this EAAccessoryManager only for notification. when USB cable connects or disconnect it can notify to the user. But I need to disconnect connection programmatically . is there any API or Tutorial available for this ? Kindly help me please.

You should probably look at IOKit. I believe IOKit works with USB device both on iOS and OS X.

You may be interested to take a look at these link: https://developer.apple.com/library/mac/#documentation/DeviceDrivers/Conceptual/USBBook/USBIntro/USBIntro.html

http://iphonedevwiki.net/index.php/IOKit.framework

Also, there is very good book on iOS/OS X kernel and I believe there is a whole chapter regarding IOKit:

http://www.amazon.com/Mac-OS-iOS-Internals-Apples/dp/1118057651

Often, when running LLDB on OSX, the debugger will sporadically crash (this just seems to be the nature of LLDB). When this happens, I get a zombie process (the one I was debugging) that looks really strange. It has parent PID = 0:

My process is called: worker

Here is the output of ps aux | grep worker

root            22785   0.0  0.0        0      0   ??  Z     3:48pm   0:00.00 (worker)

As you can see, the process has a PID, but it has no parent PID. This does nothing

sudo kill -9 <pid>

In fact, nothing short of a restart seems to get rid of it. How can I kill a process like that? I need the process to die, because it holds on to ports that I need.

The situation is that it is not possible, as a user, to clean up a zombie process; the kernel will handle it when it's ready, if possible.

As detailed in Mac OS X and iOS Internals,

it is a perfectly normal state and every process usually spends an infinitesimal amount of time, just before it can rest in peace....Parents who outlive, yet forsake their children and move on to other things, will damn the children to be stuck in the quasi-dead state of a zombie. Zombies are, for all intents and purposes, quite dead.

Since the debugger acts as the parent to its target process, which holds a network port and the debugger has crashed, it leaves the target process in the zombie state. The problem here is the crash of the debugger.

I have two questions regarding Remote Control Events on iOS:

  1. I know that music applications are registered to remote control events and then can receive such events from the iPhone's player widget. Let's say I want my app to fire such events, is that possible?
  2. How does headphones for example generate those events?

I don't know if it possible for headphone events but with private API, you can send some events like: home button press, power button press or mouse events (not tested).

You should read this book:

http://www.amazon.com/gp/product/1118057651/ref=pd_lpo_sbs_dp_ss_1?pf_rd_p=1535523702&pf_rd_s=lpo-top-stripe-1&pf_rd_t=201&pf_rd_i=0321278542&pf_rd_m=ATVPDKIKX0DER&pf_rd_r=0T2AMHJCEEKJN41YJHD5

It'll be a hard work to make this works.

Take a look at GSEvent to know how to send an event to OS (iOS).

Edit: I've found 2 event types in GSEvent.GSEventType:

kGSEventHeadsetButtonDown = 1018,
kGSEventHeadsetButtonUp = 1019,

PS: - this use private API so it will be rejected if you post this app to AppStore.

Levin had a chapter in his book Mac OS X and iOS internals: To the Apple's Core. related to The Default Freezer which can be used in Virtual Memory Management. It is stated that it is implemented in osfmk/vm/default_freezer.c

At the time of writing this was still incomplete. Does anyone know if this is implemented in newer versions of iOS? I did not find a lot of documentation for this.