Monthly Archives: July 2015

virtualbox – OS development and debugging

The Debugger for virtualbox is somewhat unknown. There are not a lot of resources online about it and it is not even visible by default. However it is a really powerful tool that makes the sensitive and low level implementation of OS much easier.

See my earlier post how to enable the debugger on a windows host.


I have followed the code in the little book of OS development and reached chapter 5 about segmentation. Segmentation is a low level task that is best suited for pure assembler which makes it very difficult to know if you did anything wrong. Or more precisely, it is difficult to know what you did wrong, since an error will generate a crash. Logging or printing to the screen will often not help you.

The debugger gives you a tool to observe your system from the outside, relatively immune from the crash and with plenty of command to show low level info such as registers and address locations.

Writing “help commands” will give you a huge list of debug commands. You can see more info about the separate commands by typing “help <command>”

Finding points of interests

The first thing to do is to find where in memory interesting code is. Looking for hex pattern is quite difficult. A better option is to look for strings used by the OS. For an example, I have the string “Good bye sir” once the kmain function is done, before the infinity jmp loop (see the OS Development).

Searching for a string can be made by sa <range> <pattern>.

If you have a string, let’s say “Good bye sir” you can search for where it is in memory with:

>sa 0 “Good bye”

The output should be something like:

VBoxDbg> sa 0 “Good bye”
%%000000000010054a: 47 6f 6f 64 20 62 79 65-20 73 69 72 2e 0a 00 00 Good bye sir….

Next we would like to see what else there is in the memory next to the “Good bye string”. Displaying byte info about a memory area is done by db <address>

Usually your code is in the nearby of strings if you have made a small assembler program for booting (but it is certainly not true for larger programs).

VBoxDbg> db 10054a
%000000000010054a: 47 6f 6f 64 20 62 79 65-20 73 69 72 2e 0a 00 00 Good bye sir….
%000000000010055a: 00 00 02 b0 ad 1b 00 00-00 00 fe 4f 52 e4 b8 be ………..OR…
%000000000010056a: ba fe ca bc 04 18 10 00-e8 89 fa ff ff eb fe 00 …………….
%000000000010057a: 00 00 14 00 00 00 00 00-00 00 01 7a 52 00 01 7c ………..zR..|
%000000000010058a: 08 01 1b 0c 04 04 88 01-00 00 1c 00 00 00 1c 00 …………….
%000000000010059a: 00 00 64 fa ff ff 64 00-00 00 00 41 0e 08 85 02 ..d…d….A….

There are no more strings after that line. Then the data below might be executable instructions. You can show what the hex is in assemble by  u32 <address>

If we take a look at the memory address we got above:

VBoxDbg> u32 10055a
%000000000010055a 00 00 add byte [eax], al
%000000000010055c 02 b0 ad 1b 00 00 add dh, byte [eax+000001badh]
%0000000000100562 00 00 add byte [eax], al
%0000000000100564 fe 4f 52 dec byte [edi+052h]
%0000000000100567 e4 b8 in AL, 0b8h
%0000000000100569 be ba fe ca bc mov esi, 0bccafebah
%000000000010056e 04 18 add AL, 018h
%0000000000100570 10 00 adc byte [eax], al
%0000000000100572 e8 89 fa ff ff call 000100000h
%0000000000100577 eb fe jmp -002h (000100577h)

As you can see, there is the “cafebabe”, a call to where we placed our kernel (“call 000100000h”) and the endless jump loop in the end “jmp -002h”.

We want to pause the program so we need to add a BP. There are several options for this. I use br <address>

I’m not sure about the  the difference between bp and br. However my system crashed if I try to use bp. I suggest that you try both.

>br 100577

If you want to see all your BP use bl

VBoxDbg> bl
0x4 e 1 r 0000000000100577 0000 (0000 to ~0)

Now we have paused our program. If we would like to execute the next command/trace we can do that by t

VBoxDbg> t
dbgf event: Single step! (rem)
eax=00000000 ebx=0002cd80 ecx=0000000d edx=0000001b esi=0002cef0 edi=0002cef1
eip=00100577 esp=00101804 ebp=00067ee0 iopl=0 nv up di pl zr na po nc
cs=0008 ds=0010 es=0010 fs=0010 gs=0010 ss=0010 eflags=00000046
0008:00100577 eb fe jmp -002h (000100577h)

Perhaps not a very useful BP since it is in the jmp loop.

There are plenty of nice dump commands, example to check your GDT like you set up in chapter 5 you can use dg

VBoxDbg> dg
0008 CodeEO Bas=00000000 Lim=fffff000 DPL=0 P NA G BIG AVL=0 L=0
0010 DataRW Bas=00000000 Lim=fffff000 DPL=0 P NA G BIG AVL=0 L=0


A problem when debugging is that you often need to set a BP before  the OS boots up, because the OS won’t halt and you will miss the code execution point. For a simple OS you can often reuse the same memory address, like 0x100577, and if you followed the OS book then 0x100000 will be the C code entry. The VM will start in a paused state, so add the BP and then resume.

virtualbox – The Little Book About OS Development

Since I want to learn more about Linux  Device Driver development I need to learn more how the OS work from scratch. There are many tutorials about the Linux kernel, but they are often too heavy. I need a guide that show me in small steps how to set things up. There is a one great tutorial:  The little book about OS development. I am really happy that the authors put up all the effort to write this and publish it for free! It is not Linux but it is still based on the same problems that most OS has to deal with during boot up.

Note that it is a bit “old school”, some techniques are not well suited for developing a real OS, how ever if you like me want to get a better understanding of multitasking, virtual memory IRQ and such it is perfect. IF you plan to make a real OS you should read beginners mistakes at the Osdev wiki, quite a lot of these faults are in the little book. Another thing to keep in mind is that most links are outdated, I found this tutorial to be very well written (down to explaining the chip architecture!)

I am a big fan of virtualbox so I decided to update the tutorial with how to use it instead of the brosh emulator that is used in the book.

You could either develop directly on a Linux host. This will save you time. However I’m on a windows host and I will use a Linux Guest to develop on. If you use Linux as a host you can skip the part about exporting the iso-file.


Let’s follow until the first step of the book: hello CAFEBABE

We need to create a guest OS that will run the compiled OS. Let’s call it guest C (as Compiled or CAFEBABE). You can optionally develop from a guest Linux OS. If you are on a host Linux this is not needed. Let’s call this OS guest L.

First create guest C:

  • No EFI ( let’s not make it complicated)
  • Don’t create any HD. We will boot from the ISO.

Optional. Set up Guest L:

  • Add a shared folder and name it osdev. Enable auto mount.
  • In guest L, add your user to vboxfs group: $adduser $USER vboxfs. Reboot.

Let’s make an export script*. Create a file create_iso. Add the gensioimage command and copy the iso to the shared folder:

genisoimage -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -A os -input-charset utf8 -quiet -boot-info-table -o os.iso iso
#copy iso to shared folder
cp os.iso /media/sf_osdev

This is a very simple script. You should probably add some error handling if an iso couldn’t be created.

*In later chapters we will use a make file. I suggest to add the cp line in that make file:

os.iso: kernel.elf
cp kernel.elf iso/boot/kernel.elf
genisoimage -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -A os -input-charset utf8 -quiet -boot-info-table -o os.iso iso
cp os.iso /media/sf_osdev/

Running the guest and debugging

Once you got the iso (if you exported from guest L it will be in osdev folder on your host) you can add it to an optic drive on your C guest. Running it is not very fun, since you can’t see any registers – we need to enable debugging.

By default debugging is disabled, see the manual. There are a couple of options to enable it. For Linux it is quite easy since it is a command line friendly OS. If you use Windows as host the easiest will be the following option (quoted from the manual):

“Set the VBOX_GUI_DBG_ENABLED or VBOX_GUI_DBG_AUTO_SHOW environment variable to true before launching the VirtualBox process. Setting these variables (only their presence is checked) is effective even when the first VirtualBox process is the VM selector window. VMs subsequently launched from the selector will have the debugger enabled.”

In Windows this is done by opening a command promt and entering the following:


Restart virtualbox and start guest C.

It is paused by default, so press ctrl  p. You will see info that the kernel is loaded. To see the 32 bit registers use the debug command: rg32:



As you can see there’s a CAFEBABE in eax.


Installing Gentoo on EFI and virtualbox is tough!

The problem is because the gentoo CD doesn’t come with a EFI boot, so you have to start from MBR. Well, it turns out that it’s not possible to change the EFI settings if you have booted through MBR.

That means you have to install gentoo without EFI support first and make a boot partition ready for the EFI boot loader. How do you do that without being able to configure EFI?

If you are just looking for the solution go to the short version.

The Fix – Long Version

First of all in Gentoo you must enable EFI stub so that the kernel can be executed as a boot loader (awesome functionality!).

Secondly if EFI has nothing specified it will look for a file named “boot<arch>.efi” in \efi\boot. So in most cases ‘\efi\boot\bootx64amd.efi‘. Notice that since EFI requires FAT the path uses backslash rather than forward slash (/). So you need to copy the kernel there and rename it.

After rebooting (and turning on EFI in virtualbox) the kernel loads. But panics:

VFS: Cannot open root device (null) or unknown-block(0 0): error -6
Please append a correct “root=” boot option; here are the available partitions:
(omitted sda partitions)
Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

What does this mean? At first I thought it was because I didn’t set any EFI variables. It is possible to do so through the EFI Shell. You can start this by mashing F12 when virtualbox is booting up a VM. From here there are many commands you can run to tweak your boot up. And written by some drunk lemurs, really wtf Intel!

Let’s have a look at some useful commands:

ram -r: shows the boot options.

bcfg: adds boot options. Example is bcfg boot add <option nr> <efi file> “<label>” . In my case this command would look like blk3:\efi\boot\bootx64.efi

I never figured out how to add kernel arguments (there’s a “-opt” option to bcfg).

This did not help, I still got kernel panic. My second thought was that it was because I did not have the driver for the HDD. I recompiled the kernel with anything that rhymed with EFI,SATA ,PIIX or ACHI. It didn’t help. The live  CD could find the drives, so I took every module it used and put into /etc/conf.d/modules by lsmod | gawk ‘{print $1}’. It didn’t help (I got quite a lot of warnings though).

Thinking about it, since the kernel actually got loaded it must have found the device. The problem is probably the ‘correct “root=”…’ message.

It turns out that you can set the root partition by a kernel command line option. I had no idea, but this is something boot loaders usually do. I heard you could also add these options when you compile the kernel, but I don’t want to recompile my kernel. Instead I booted up in UEFI Shell again and started the efi file with a root option:

>blk3:\efi\root\bootx64amd.efi root=/dev/sda2

It worked! Gentoo boots, and I got a warm and nice feeling in my stomach. Since it has booted from EFI it should be possible to set the parameters through efibootmgr:

$efibootmgr -c -d /dev/sda -p 1 -L “gentoo Linux” -l ‘\efi\boot\bootx64amd.efi’  root=/dev/sda2
$efibootmngr -v

The output shows it worked. But after reboot still kernel panic. Checking the boot options in UEFI Shell I can see that there’s no record saved. Is it a bug in efibootmgr? I spent a few more hours trying to add options with bcfg when I realized there’s a really nice way to add boot options. Don’t boot the shell, instead enter Boot Maintenance Management. From here it’s very easy to add a record with extra arguments.

Starting is fine, but as soon as I reboot the EFI variables will be wiped! This must be a bug (feature?) in virtualbox.

There is one final resort. If EFI does not find an efi file in ESP it will try to execute startup.nsh in the efi\boot folder. Since it will execute the efi file first it’s important to rename it. I renamed mine to gentoo.efi.

Create a startup.nhs file with the UEFI Shell command:


save it and reboot.

Finally Gentoo can automatically boot up in EFI mode!

The fix – short version

  • First install gentoo and create a FAT32 boot partition (and all other things needed for EFI, such as GPT and ESP). Don’t forget to update fstab.
  • On first reboot enter the boot menu by pressing F12.
  • Go to Boot Maintenance manager -> Boot Options -> add boot options
  • Select the boot partition, probably the only one with a GUID. Find the location of your efi file (efi\boot\bootxamd64.efi)
  • Add description (any thing you like). Input Option Data: root=/dev/sda2
  • Commit changes and exit. Go back and select boot manager and select your created boot option.
  • Once gentoo boots up you can try to add a record from efibootmgr. If it doesn’t save after reboot continue with the steps below.
  • Rename the efi file to something like /efi/boot/gentoo.efi
  • Create startup.nsh in the same folder and add the UEFI Shell path to the efi file. Example: blk3:\efi\boot\gentoo.efi
  • Reboot.

Archade Games

A while ago I stumbled upon a post about the difference between distros. One user explained Arch as:

“It is a practical joke. You follow the instructions on the wiki, and it just says at the end: congrats and thanks for playing. try again for a better score.”

I had tried a few Ubuntu forks and even CentOS at that time, but still I could not understand the post. Apparently the Linux vets in the thread thought it was an incredible funny joke but I was left clueless.

Until today. I decided to find out what this Arch Linux is all about. I made up my mind to install it and to make it more interesting I would not give up until I had:

  • Using UEFI for booting
  • Being able to run a desktop environment
  • Understand this funny joke about some “wiki”.

Game on!

Level 1 – Welcome.

Downloading Arch and setting up virtualbox was as easy as all other distros I’ve tried. Enabling EFI is just checking the box under motherboard at virtualbox. After a few minutes I was already booting and I would soon be a few clicks away from installing Arch.

But wait a minute… isn’t there something missing from the menu?


Where is the install option?

Starting Arch Linux archiso just boot you into a Console. What am I supposed to do from here? Not even my keyboard layout is correct.

Level 2 – That wiki

The first hits on google for “install arch” takes you to the Arch wiki. While even the beginners guide is a few pages long, it is excellent written for an inexperienced Linux user like me. I don’t think I ever learned so much about Linux in a single day as when I tried to make my boot partition (Wait, what, so my drive is just a file? Aaaaah!).

With that said I got a bit terrified when I even had to load my keyboard settings. And that was not so easy, doing like the wiki said “loadkeys <country code>” did not work, but after a while I found out that the correct usage (for swedish) is #loadkeys i386/qwerty/

“Arch Linux, a lightweight and flexible Linux® distribution that tries to Keep It Simple.”

That ‘simple’ stings in my pride every time I have to check the wiki.

Level 3 – Let’s get this parted started

As this was the first time I did partitioning it did take me a while to understand how to set up a UEFI boot partition. But this is one of the reason why I love VM: I am quite ruthless to my poor computer and I rather try what I think is right than keep reading. This often makes me learn faster – however you don’t really want to do this when you are partitioning your HD. At least not if you have another OS installed like me. With VM I don’t need to care.
I might write in another post how to set up a UEFI boot with virtualbox, it had some other nice effects like increased screen resolution in console mode. The trick is to create a partition with GPT and fat32.

Level 4 – pacstrap

After mounting and chroot ( another new concept I learned today) you do a pacstrap. pacstrap basically downloads Linux to your newly set up partitions. What impress me is that with the -i options you can even choose what packages you want to include, Arch is really flexible! Then time to reboot. Linux finally!

>You are now being dropped into an emergency shell

Ok, perhaps not yet. But after a few more hours I accomplished my first goal.

Level 5 – Desktop

I like lightweight desktops so I decided I’d try xfce. First you had to install xorg-server and some vesa drivers (since I’m on virtualbox I had to use a generic driver). However trying to start the desktop just generated the error “Can’t find any screens”.

I got a bit worried that it would not be possible to load xfce with generic drivers, but as I troubleshooting I finally found the error: failed to load module vboxvideo. Well of course, I hadn’t installed the guest additions! Installing the guest additions was a bit different than for Ubuntu, but just as easy.  Finally the desktop starts… I made it! I beat the game! Or…?



I recommend everyone who have been using Linux as desktop for a year or two to install Arch. You will learn a lot about Linux that I think is very difficult to do without practical experience. And it is fun! Well, like all games you do get tired of it after the first few runs, but right now there’s many more things to explore.

Virtualbox – Web Server ABC

Do you want to learn to make a website but don’t want to install http server on your host? In this guide I will show how to set up a guest OS and some popular tools for web servers.

virtualbox local network – see previous post

60 min

Table of content


Without any doubt the most common http server is apache. What OS that is best to use it with is a bit more divided. In the Linux world the the most popular the free distros is probably CentOS. It’s based on RedHat Enterprise Linux which makes it a bit different from the Ubuntu clones we’ve used in the previous posts, and because we will make a web server we will not use any desktop environment. In matter fact to learn as much as possible we will install the minimum CentOS! Because we do it in a guest we can still use web browser in our host, something that is really nice when you need to ask google for help.
It might seem bothersome to have the server on a guest since you need to set up the VM environment properly to use network, but these days virtual servers is quite common: they use less power and you don’t have to turn your apartment into a storage room for computers.

Step 0 – Install CentOS

Download the iso from I use CentOS-7-x86_64-DVD-1503-01.iso in this guide.

CentOS will give you a warning if you don’t have more than one processor:
Important: In CentOS 7, single threaded, single CPU 64-bit physical systems are unsupported

During the installation do the following options:

Encrypt my data:  enable.

Minimal install

Make sure you turn your connection on.

Create a user called ‘admin’ with administration privilege. The installation on my machine was really resource heavy and lags a lot, I have never experienced it with other distros.

Step 1 – Installing the environment

First thing is to update the system.
#yum update
#yum upgrade

If you have only used Ubuntu then you might wonder what yum is. It’s the package manager for RHEL (well, until recently when it switched to dnf). The basic usage is similar to apt-get.

We will only use console so we need a console based editor:
#yum install vim

Since we are going console mode it would be nice to multiplex. There are two options, tmux and screen . In this tutorial I will use screen.
The alternative is of course to use the different virtual consoles [ctrl alt F1 -F6], but you can do a lot of other nice things with screen so it is recommended.
#yum install screen

Now let’s change the default ssh port. In /etc/ssh/sshd_config find ‘#port 22’ and remove the ‘#’ and set the port to 50022:
port 50022

Find PermitRootLogin and set it to no
PermitRootLogin no

Save and exit.
You will need to restart the ssh server for the changes to take effect:
#systemctl restart sshd

Go a head and try it if you want:
$ssh admin@localhost
ssh: connection to host localhost port 22: Connection refused
$ssh root@localhost -p 50222
Permission denied, please try again
$ssh admin@localhost -p 50222

Step 2 – setting up network

We will open up one more guest. This guest needs a desktop environment since we will use a web browser. I will use a Xubuntu guest. Let’s call the CentOS guest C and the Xubuntu guest X.
First add both guest C and X to a local network like we did previous. Let C still have it’s NAT adapter, but remove the cable.

Get the ip of C:
$ip addr

Make sure we can ssh from X to C:
$ssh admin@ -p 50022
ssh: connect to host port 50022: No route to host

Strange, let’s see if we can ping C:
4 packets transmitted, 4 received, 0% packet loss, time 3005ms

It seem as we can reach the host. The port seem to be closed. Something is blocking it, most likely a firewall.

Step 3 – iptables

Many Linux distros comes with a firewall/IP filter called iptables*. It’s a really complex and advanced module and I will only cover some basics in this tutorial. If you got some time check out this awesome tutorial.

Make sure your NAT adapter is unplugged for guest C. Now let’s turn of iptables:
#iptables -P INPUT ACCEPT
#iptables -F

The first line allows all input (so make sure internet adapter is unplugged!). The second line flushes the table.

Retry to ssh from X to C.
It works. Now we need to add some real rules to iptables.
We want to allow ssh connections on port 50022, and since we will set up a web server we will also open port 80.

#iptables -P INPUT DROP
This line adds a policy to drop all incoming traffic.
We still want to accept http and ssh so we need to add some exceptions:
#iptables -A INPUT -p tcp –dport 50022 -j ACCEPT
#iptables -A INPUT -p tcp –dport 80 -j ACCEPT
Both this lines makes the firewall accept port 50022 and 80 for tcp traffic.

#iptables -P FORWARD DROP
Drop all forwarding
#iptables -P OUTPUT DROP
Allow all outgoing traffic

Lastly we will allow incoming traffic for already established or related traffic.
#iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

* CentOS 7 uses firewallD – but I still had to open the port in iptables.

Step 4 -apache

Connect your adapter to internet again and install apache:
#yum install httpd
Then start the service:
#systemctl start httpd.service

Now from guest X open the web browser and enter the ip address of C.
You should see the default test page.




It’s quite easy to create a web server on a virtual machine. Usually when you use a virtual machine for hosting you use the VM in a headless state and connect to it through ssh – there’s no point of having a GUI at all. This requires static IP and that the host forwards the traffic to the guest though. We will have a look in upcoming posts of some alternatives to connect to your virtual web server from the outside.