virtualbox – OS development and debugging

The Debugger for virtualbox is somewhat unknown. There are not a lot of resources online about it and it is not even visible by default. However it is a really powerful tool that makes the sensitive and low level implementation of OS much easier.

See my earlier post how to enable the debugger on a windows host.

Introduction

I have followed the code in the little book of OS development and reached chapter 5 about segmentation. Segmentation is a low level task that is best suited for pure assembler which makes it very difficult to know if you did anything wrong. Or more precisely, it is difficult to know what you did wrong, since an error will generate a crash. Logging or printing to the screen will often not help you.

The debugger gives you a tool to observe your system from the outside, relatively immune from the crash and with plenty of command to show low level info such as registers and address locations.

Writing “help commands” will give you a huge list of debug commands. You can see more info about the separate commands by typing “help <command>”

Finding points of interests

The first thing to do is to find where in memory interesting code is. Looking for hex pattern is quite difficult. A better option is to look for strings used by the OS. For an example, I have the string “Good bye sir” once the kmain function is done, before the infinity jmp loop (see the OS Development).

Searching for a string can be made by sa <range> <pattern>.

If you have a string, let’s say “Good bye sir” you can search for where it is in memory with:

>sa 0 “Good bye”

The output should be something like:

VBoxDbg> sa 0 “Good bye”
%%000000000010054a: 47 6f 6f 64 20 62 79 65-20 73 69 72 2e 0a 00 00 Good bye sir….

Next we would like to see what else there is in the memory next to the “Good bye string”. Displaying byte info about a memory area is done by db <address>

Usually your code is in the nearby of strings if you have made a small assembler program for booting (but it is certainly not true for larger programs).

VBoxDbg> db 10054a
%000000000010054a: 47 6f 6f 64 20 62 79 65-20 73 69 72 2e 0a 00 00 Good bye sir….
%000000000010055a: 00 00 02 b0 ad 1b 00 00-00 00 fe 4f 52 e4 b8 be ………..OR…
%000000000010056a: ba fe ca bc 04 18 10 00-e8 89 fa ff ff eb fe 00 …………….
%000000000010057a: 00 00 14 00 00 00 00 00-00 00 01 7a 52 00 01 7c ………..zR..|
%000000000010058a: 08 01 1b 0c 04 04 88 01-00 00 1c 00 00 00 1c 00 …………….
%000000000010059a: 00 00 64 fa ff ff 64 00-00 00 00 41 0e 08 85 02 ..d…d….A….

There are no more strings after that line. Then the data below might be executable instructions. You can show what the hex is in assemble by  u32 <address>

If we take a look at the memory address we got above:

VBoxDbg> u32 10055a
%000000000010055a 00 00 add byte [eax], al
%000000000010055c 02 b0 ad 1b 00 00 add dh, byte [eax+000001badh]
%0000000000100562 00 00 add byte [eax], al
%0000000000100564 fe 4f 52 dec byte [edi+052h]
%0000000000100567 e4 b8 in AL, 0b8h
%0000000000100569 be ba fe ca bc mov esi, 0bccafebah
%000000000010056e 04 18 add AL, 018h
%0000000000100570 10 00 adc byte [eax], al
%0000000000100572 e8 89 fa ff ff call 000100000h
%0000000000100577 eb fe jmp -002h (000100577h)

As you can see, there is the “cafebabe”, a call to where we placed our kernel (“call 000100000h”) and the endless jump loop in the end “jmp -002h”.

We want to pause the program so we need to add a BP. There are several options for this. I use br <address>

I’m not sure about the  the difference between bp and br. However my system crashed if I try to use bp. I suggest that you try both.

>br 100577

If you want to see all your BP use bl

VBoxDbg> bl
0x4 e 1 r 0000000000100577 0000 (0000 to ~0)

Now we have paused our program. If we would like to execute the next command/trace we can do that by t

VBoxDbg> t
dbgf event: Single step! (rem)
eax=00000000 ebx=0002cd80 ecx=0000000d edx=0000001b esi=0002cef0 edi=0002cef1
eip=00100577 esp=00101804 ebp=00067ee0 iopl=0 nv up di pl zr na po nc
cs=0008 ds=0010 es=0010 fs=0010 gs=0010 ss=0010 eflags=00000046
0008:00100577 eb fe jmp -002h (000100577h)

Perhaps not a very useful BP since it is in the jmp loop.

There are plenty of nice dump commands, example to check your GDT like you set up in chapter 5 you can use dg

VBoxDbg> dg
0008 CodeEO Bas=00000000 Lim=fffff000 DPL=0 P NA G BIG AVL=0 L=0
0010 DataRW Bas=00000000 Lim=fffff000 DPL=0 P NA G BIG AVL=0 L=0

 

A problem when debugging is that you often need to set a BP before  the OS boots up, because the OS won’t halt and you will miss the code execution point. For a simple OS you can often reuse the same memory address, like 0x100577, and if you followed the OS book then 0x100000 will be the C code entry. The VM will start in a paused state, so add the BP and then resume.

Leave a Reply

Your email address will not be published. Required fields are marked *