One thing I've always wanted to do is develop my very own operating system (not necessarily fancy like Linux or Windows, but better than a simple boot loader which I've already done).
I'm having a hard time finding resources/guides that take you past writing a simple "Hello World" OS.
I know lots of people will probably recommend I look at Linux or BSD; but the code base for systems like that is (presumably) so big that I wouldn't know where to start.
Update: To make it easier for people who land on this post through Google here are some OS development resources:
Writing Your Own Operating System (Thanks Adam)
Linux From Scratch (Thanks John)
SharpOS (C# Operating System) (Thanks lomaxx)
BonaFide (Thanks Steve)
Bran (Thanks Steve)
Roll your own toy UNIX-clone OS (Thanks Steve)
I found a nice resource named MikeOS, "MikeOS is a learning tool to demonstrate how simple OSes work. It uses 16-bit real mode for BIOS access, so that it doesn't need complex drivers"
I found some resources at Freebyte's Guide to...Free and non-free Operating Systems that links to kits such as OSKit and ExOS library. These seem super useful in getting started in OS development.
Dinah provided some useful Stack Overflow discussion of aspiring OS developers: Roadblocks in creating a custom operating system discusses what pitfalls you might encounter while developing an OS and OS Development is a more general discussion.
LB provided a link to the Pintos Project, an education OS designed for students learning OS development.
Updated 7/27/09 (Still going strong!)
I stumbled upon an online OS course from Berkley featuring 23 lectures.
I found the slides and other materials to go along with the online Berkeley lectures listed above.
All questions tagged osdev on stackoverflow
OS/161 is an academic OS written in c that runs on a simulated hardware. This OS is similar in Nachos. Thanks Novelocrat!
Linux Kernel Development by Robert Love is suggested by Anders. It is a "widely acclaimed insider's look at the Linux kernel."
Thanks Tim S. Van Haren for telling us about Cosmos, an OS written entirely in c#.
tgiphil tells us about Managed Operating System Alliance (MOSA) Framework, "a set of tools, specifications and source code to foster development of managed operating systems based on the Common Intermediate Language."
kerneltrap.org is no longer available. The linux kernel v0.01 is available from kernel.org
Updated 12/21/2012 A basic OS development tutorial designed to be a semester's project. It guides you through to build an OS with basic components. Very good start for beginners. Related paper. Thanks Srujan!
Writing a Simple Operating System From Scratch. Thanks James Moore!
How to make a computer operating system Thanks ddtoni!
Updated Sept 12 2016
Updated Dec 10 2016
Writing a Simple Operating System —from Scratch (thank you @Tyler C)
There are a lot of links after this brief overview of what is involved in writing an OS for the X86 platform.
The link that appears to be most promising (www.nondot.org/sabre/os/articles) is no longer available, so you'll need to poke through the Archive.org version to read it.
At the end of the day the bootloader takes the machine code of the kernel, puts it in memory, and jumps to it. You can put any machine code in the kernel that you want, but most C programs expect an OS so you'll need to tell your compiler that it won't have all that, or the bootloader has to create some of it.
The kernel then does all the heavy lifting, and I suspect it's the example kernel you want. But there's a long way to go between having a kernel that says, "Hello world" to having a kernel that loads a command interpretor, provides disk services, and loads and manages programs.
You might want to consider subscribing to ACM to get access to their older literature - there are lots of articles in the late 80's and early 90's in early computing magazines about how to create alternative OSs. There are likely books that are out of print from this era as well. You might be able to get the same information for free by looking up the indexes of those magazines (which are available on that site - click "index" near the magazine name) and then asking around for people with a copy.
Lastly, I know that usenet is dead (for so sayeth the prophets of internet doom) but you'll find that many of the craggy old experts from that era still live there. You should search google groups (they have dejanews's old repository) and I expect you'll find many people asking the same questions a decade or 1.5 ago that you're asking now. You may even run across Linus Torvalds' many queries for help as he was developing linux originally. If searches don't bring anything up, ask in the appropriate newsgroup (probably starts with comp.arch, but search for ones with OS in the name).
I would like to learn how to write device drivers because I think it would be fun. I use a Mac OS X Macbook, but I also have an Ubuntu machine (running on a Mac Min). I am pretty familiar with C and currently am reading this book. I have found some links online such as Mac Dev Center. I am doing this because it would be fun. I think there would be real gratification to see hardware operate because of software I wrote.
I guess what I would like is some tips or advice and guidance, and does anyone know of a list of devices that don't have drivers or can I write a driver for something that's already supported (would prefer the former so I'm actually providing value). What's a good device to get started with? Am I biting off more than I can chew? I'm not afraid of low level programming or assembly or whatever amount of effort is required. I'd like a challenge really!
The Linux Kernel Module Programming Guide is another good resource.
I'd like to gain better knowledge of operating system internals. Process management, memory management, and stuff like that.
I was thinking of learning by getting to know either linux or BSD kernel.
Which one kernel is better for learning purposes?
What's the best place to start?
Can you recommend any good books?
There's no substitute for diving into the code. Try to find a driver or subsystem that you're interested in and poke around with it. With tools like VMware Workstation it's super easy to make whatever changes you want, snapshot the VM, and run your modified kernel. If the kernel panics on boot, who cares? Just jump back to the snapshot and fix the problem.
For books, I strongly recommend Linux Kernel Development by Robert Love. It's a wonderfully written book -- lots of information, organized sanely, and humorous... not dry reading at all.
Noting the lack of BSDs here, I figured I'd chip in:
I haven't taken any of the courses myself, but I've heard Marshall Kirk McKusick speak on other occasions, and he is really good at what he does.
And of course the BSD man pages, which are an excellent resource as they are maintained to a far greater extent than your average Linux man-page. Take for instance the uvm(9) man-page, describing the virtual memory interface in OpenBSD.
Not quite related, but I'll also recommend the video History of the Berkeley Software Distributions as it gives a nice introduction to the BSD parts of the UNIX history and culture as well as plenty of hilarious anectodes from back when.
I had previously bought these books on recommendation for the same purpose but I never got to studying them myself so only take them as second-hand advice.
One thing to note though, if you pick Minix, it is designed for learning. It is a microkernel, while Linux and BSD are a monolithic kernel, so what you learn may not be 100% translatable to be able to work with Linux or BSD, but you can still gain a lot out of it, without having to process quite as much information.
As a side note, if you've read Just for Fun, Linus actually was playing with Minix before he wrote Linux, but it just wasn't enough for his purposes.
What is a good resource to get started with Windows file system driver development for a newbie?
or any newer book you can find by an Art Baker. I read his NT Device Driver book about 10 years ago, and it finally made everything clear.
BTW, the books from 10 years ago or more/less still valid. You can't use the examples, but the model has basically not changed - just gotten more complex in typical M$ fashion. The IRP stuff is all still valid.
The OSR stuff is good - but expensive... I think for a full understanding of the whole design Baker can't be beat. Also, anyone reading this just trying to learn Windows drivers - I would avoid the NTFS stuff... its super complicated, and has nothing to do with what you need to accomplish a simple USB driver, or even a DMA device.
I'm porting / debugging a device driver (that is used by another kernel module) and facing a dead end because dma_sync_single_for_device() fails with an kernel oops.
I have no clue what that function is supposed to do and googling does not really help, so I probably need to learn more about this stuff in total.
The question is, where to start?
Oh yeah, in case it is relevant, the code is supposed to run on a PowerPC (and the linux is OpenWRT)
EDIT: On-line resources preferrable (books take a few days to be delivered :)
Essential Linux Device Drivers ( Only the first 4 - 5 chapters )
the Linux Cross Reference ( Searchable Kernel Source for all Kernels )
dma_sync_single_range_for_cpu a little further up in the file and this is the source documentation ( I assume that even though this is for arm the interface and behavior are the same ):
/** 380 * dma_sync_single_range_for_cpu 381 * @dev: valid struct device pointer, or NULL for ISA and EISA-like devices 382 * @handle: DMA address of buffer 383 * @offset: offset of region to start sync 384 * @size: size of region to sync 385 * @dir: DMA transfer direction (same as passed to dma_map_single) 386 * 387 * Make physical memory consistent for a single streaming mode DMA 388 * translation after a transfer. 389 * 390 * If you perform a dma_map_single() but wish to interrogate the 391 * buffer using the cpu, yet do not wish to teardown the PCI dma 392 * mapping, you must call this function before doing so. At the 393 * next point you give the PCI dma address back to the card, you 394 * must first the perform a dma_sync_for_device, and then the 395 * device again owns the buffer. 396 */
I've been reading about Linux Kernel development / device drivers and all of the books I've looked at use the pre-2.6.20 workqueue interface. This includes the Linux Kernel Module Programming Guide, Robert Love's Linux Kernel Development, and Sreekrishnan Venkateswaran's Essential Linux Device Drivers which was published in April 2008!
Does anyone have a guide to using the "new" 2.6.20+ workqueue interface?
Just for reference, Ubuntu 8.04 which I'm developing on at the moment, was released over a year and a half ago and includes kernel 2.6.24.
stsquad's answer led me to this guide to porting code to the new 2.6.20 workqueue's.
You can't go wrong with the excellent API articles on LWN (http://lwn.net/Kernel/Index/) which document a lot of the new APIs as they go into the kernel. There is a whole section devoted to Workqueues which is worth a read.
Workqueues have been in the kernel for a while so I assume your thinking about the recent-ish rework of the API.
Whilst I'm at it I can heartily recommend a subscription to LWN. All their articles are available for free after a week or so but it's worth supporting the high quality writing, especially if your an aspiring kernel hacker.
How to write char device drivers in Linux?
My favorite book for learning how the kernel works, BY FAR (and I've read most of them) is:
This book is fairly short, read it first, then read the O'Reilly book on drivers.
Is there any book/resource that one can refer to, to be able to write programs at kernel/system level.. I'm looking for a programming book that could serve as a guide to write kernel codes / system level programming etc.. I have Tannenbaum's Design and Implementation. It addresses theoretical aspects well .But a book that teaches programming in such topics would be helpful. I want to be capable of implementing thread library, scheduler et al ..
I would like to suggest both the books by Robert Love. I've read a bit of the first one and its excellent. The latter was recommended by a friend.
I am looking for a good primer or technical description of the System Call mechanism that is used by operating systems to transition from user space to the kernel to invoke functions such as "open", "read", "write", etc...
Is there anything other than the Wikipedia entry? Websites, pdfs, books, source code, all are welcome :)
Well for source code, there are plenty of open source kernels to dive into.
As for books, Robert Love's book on the Linux kernel is very informative.
I have been programming in C for a couple of years in Linux. Now I want to work on linux kernel and contribute to kernel, if possible. I have been looking on the internet for the information about a starting point but I couldn't find one. So can any one please tell me where to begin?
I'm not a kernel person at all but I work with a lot of people who are very much into the kernel. They all recommend Linux Kernel Development by Robert Love as a good book on the subject. I've bought the book myself and plan to, in my copious free time, go through it.
The Kernel Mentors mailing list related resources at http://kernelnewbies.org/KernelMentors is also a good place to look at.
How to write kernel code in C ? Where can we look forward to learn more of writing kernel code ? I want to know writing programs in C that can be used to modify my kernel. How do i do that ? What resources can i look upto ?