Category Archives: Virtual Machines

UEFI Rant

Installing Gentoo on EFI and virtualbox is tough!

The problem is because the gentoo CD doesn’t come with a EFI boot, so you have to start from MBR. Well, it turns out that it’s not possible to change the EFI settings if you have booted through MBR.

That means you have to install gentoo without EFI support first and make a boot partition ready for the EFI boot loader. How do you do that without being able to configure EFI?

If you are just looking for the solution go to the short version.

The Fix – Long Version

First of all in Gentoo you must enable EFI stub so that the kernel can be executed as a boot loader (awesome functionality!).

Secondly if EFI has nothing specified it will look for a file named “boot<arch>.efi” in \efi\boot. So in most cases ‘\efi\boot\bootx64amd.efi‘. Notice that since EFI requires FAT the path uses backslash rather than forward slash (/). So you need to copy the kernel there and rename it.

After rebooting (and turning on EFI in virtualbox) the kernel loads. But panics:

VFS: Cannot open root device (null) or unknown-block(0 0): error -6
Please append a correct “root=” boot option; here are the available partitions:
(omitted sda partitions)
Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

What does this mean? At first I thought it was because I didn’t set any EFI variables. It is possible to do so through the EFI Shell. You can start this by mashing F12 when virtualbox is booting up a VM. From here there are many commands you can run to tweak your boot up. And written by some drunk lemurs, really wtf Intel!

Let’s have a look at some useful commands:

ram -r: shows the boot options.

bcfg: adds boot options. Example is bcfg boot add <option nr> <efi file> “<label>” . In my case this command would look like blk3:\efi\boot\bootx64.efi

I never figured out how to add kernel arguments (there’s a “-opt” option to bcfg).

This did not help, I still got kernel panic. My second thought was that it was because I did not have the driver for the HDD. I recompiled the kernel with anything that rhymed with EFI,SATA ,PIIX or ACHI. It didn’t help. The live  CD could find the drives, so I took every module it used and put into /etc/conf.d/modules by lsmod | gawk ‘{print $1}’. It didn’t help (I got quite a lot of warnings though).

Thinking about it, since the kernel actually got loaded it must have found the device. The problem is probably the ‘correct “root=”…’ message.

It turns out that you can set the root partition by a kernel command line option. I had no idea, but this is something boot loaders usually do. I heard you could also add these options when you compile the kernel, but I don’t want to recompile my kernel. Instead I booted up in UEFI Shell again and started the efi file with a root option:

>blk3:\efi\root\bootx64amd.efi root=/dev/sda2

It worked! Gentoo boots, and I got a warm and nice feeling in my stomach. Since it has booted from EFI it should be possible to set the parameters through efibootmgr:

$efibootmgr -c -d /dev/sda -p 1 -L “gentoo Linux” -l ‘\efi\boot\bootx64amd.efi’  root=/dev/sda2
$efibootmngr -v

The output shows it worked. But after reboot still kernel panic. Checking the boot options in UEFI Shell I can see that there’s no record saved. Is it a bug in efibootmgr? I spent a few more hours trying to add options with bcfg when I realized there’s a really nice way to add boot options. Don’t boot the shell, instead enter Boot Maintenance Management. From here it’s very easy to add a record with extra arguments.

Starting is fine, but as soon as I reboot the EFI variables will be wiped! This must be a bug (feature?) in virtualbox.

There is one final resort. If EFI does not find an efi file in ESP it will try to execute startup.nsh in the efi\boot folder. Since it will execute the efi file first it’s important to rename it. I renamed mine to gentoo.efi.

Create a startup.nhs file with the UEFI Shell command:

blk3:\efi\boot\gentoo.efi

save it and reboot.

Finally Gentoo can automatically boot up in EFI mode!

The fix – short version

  • First install gentoo and create a FAT32 boot partition (and all other things needed for EFI, such as GPT and ESP). Don’t forget to update fstab.
  • On first reboot enter the boot menu by pressing F12.
  • Go to Boot Maintenance manager -> Boot Options -> add boot options
  • Select the boot partition, probably the only one with a GUID. Find the location of your efi file (efi\boot\bootxamd64.efi)
  • Add description (any thing you like). Input Option Data: root=/dev/sda2
  • Commit changes and exit. Go back and select boot manager and select your created boot option.
  • Once gentoo boots up you can try to add a record from efibootmgr. If it doesn’t save after reboot continue with the steps below.
  • Rename the efi file to something like /efi/boot/gentoo.efi
  • Create startup.nsh in the same folder and add the UEFI Shell path to the efi file. Example: blk3:\efi\boot\gentoo.efi
  • Reboot.

Archade Games

A while ago I stumbled upon a post about the difference between distros. One user explained Arch as:

“It is a practical joke. You follow the instructions on the wiki, and it just says at the end: congrats and thanks for playing. try again for a better score.”

I had tried a few Ubuntu forks and even CentOS at that time, but still I could not understand the post. Apparently the Linux vets in the thread thought it was an incredible funny joke but I was left clueless.

Until today. I decided to find out what this Arch Linux is all about. I made up my mind to install it and to make it more interesting I would not give up until I had:

  • Using UEFI for booting
  • Being able to run a desktop environment
  • Understand this funny joke about some “wiki”.

Game on!

Level 1 – Welcome.

Downloading Arch and setting up virtualbox was as easy as all other distros I’ve tried. Enabling EFI is just checking the box under motherboard at virtualbox. After a few minutes I was already booting and I would soon be a few clicks away from installing Arch.

But wait a minute… isn’t there something missing from the menu?

minmenu

Where is the install option?

Starting Arch Linux archiso just boot you into a Console. What am I supposed to do from here? Not even my keyboard layout is correct.

Level 2 – That wiki

The first hits on google for “install arch” takes you to the Arch wiki. While even the beginners guide is a few pages long, it is excellent written for an inexperienced Linux user like me. I don’t think I ever learned so much about Linux in a single day as when I tried to make my boot partition (Wait, what, so my drive is just a file? Aaaaah!).

With that said I got a bit terrified when I even had to load my keyboard settings. And that was not so easy, doing like the wiki said “loadkeys <country code>” did not work, but after a while I found out that the correct usage (for swedish) is #loadkeys i386/qwerty/se-lat6.map.gz

“Arch Linux, a lightweight and flexible Linux® distribution that tries to Keep It Simple.”

That ‘simple’ stings in my pride every time I have to check the wiki.

Level 3 – Let’s get this parted started

As this was the first time I did partitioning it did take me a while to understand how to set up a UEFI boot partition. But this is one of the reason why I love VM: I am quite ruthless to my poor computer and I rather try what I think is right than keep reading. This often makes me learn faster – however you don’t really want to do this when you are partitioning your HD. At least not if you have another OS installed like me. With VM I don’t need to care.
I might write in another post how to set up a UEFI boot with virtualbox, it had some other nice effects like increased screen resolution in console mode. The trick is to create a partition with GPT and fat32.

Level 4 – pacstrap

After mounting and chroot ( another new concept I learned today) you do a pacstrap. pacstrap basically downloads Linux to your newly set up partitions. What impress me is that with the -i options you can even choose what packages you want to include, Arch is really flexible! Then time to reboot. Linux finally!

>You are now being dropped into an emergency shell

Ok, perhaps not yet. But after a few more hours I accomplished my first goal.

Level 5 – Desktop

I like lightweight desktops so I decided I’d try xfce. First you had to install xorg-server and some vesa drivers (since I’m on virtualbox I had to use a generic driver). However trying to start the desktop just generated the error “Can’t find any screens”.

I got a bit worried that it would not be possible to load xfce with generic drivers, but as I troubleshooting I finally found the error: failed to load module vboxvideo. Well of course, I hadn’t installed the guest additions! Installing the guest additions was a bit different than for Ubuntu, but just as easy.  Finally the desktop starts… I made it! I beat the game! Or…?

gameover_small_g

Conclusion

I recommend everyone who have been using Linux as desktop for a year or two to install Arch. You will learn a lot about Linux that I think is very difficult to do without practical experience. And it is fun! Well, like all games you do get tired of it after the first few runs, but right now there’s many more things to explore.

Virtualbox – Web Server ABC

Do you want to learn to make a website but don’t want to install http server on your host? In this guide I will show how to set up a guest OS and some popular tools for web servers.

Requirements:
virtualbox local network – see previous post

Time:
60 min

Table of content

Introduction


Without any doubt the most common http server is apache. What OS that is best to use it with is a bit more divided. In the Linux world the the most popular the free distros is probably CentOS. It’s based on RedHat Enterprise Linux which makes it a bit different from the Ubuntu clones we’ve used in the previous posts, and because we will make a web server we will not use any desktop environment. In matter fact to learn as much as possible we will install the minimum CentOS! Because we do it in a guest we can still use web browser in our host, something that is really nice when you need to ask google for help.
It might seem bothersome to have the server on a guest since you need to set up the VM environment properly to use network, but these days virtual servers is quite common: they use less power and you don’t have to turn your apartment into a storage room for computers.

Step 0 – Install CentOS


Download the iso from https://www.centos.org/download/. I use CentOS-7-x86_64-DVD-1503-01.iso in this guide.

CentOS will give you a warning if you don’t have more than one processor:
Important: In CentOS 7, single threaded, single CPU 64-bit physical systems are unsupported

During the installation do the following options:

INSTALLATION DESTINATION
Encrypt my data:  enable.

SOFTWARE SELECTION
Minimal install

NETWORK & HOST NAME
Make sure you turn your connection on.

Create a user called ‘admin’ with administration privilege. The installation on my machine was really resource heavy and lags a lot, I have never experienced it with other distros.

Step 1 – Installing the environment


First thing is to update the system.
#yum update
#yum upgrade

If you have only used Ubuntu then you might wonder what yum is. It’s the package manager for RHEL (well, until recently when it switched to dnf). The basic usage is similar to apt-get.

We will only use console so we need a console based editor:
#yum install vim

Since we are going console mode it would be nice to multiplex. There are two options, tmux and screen . In this tutorial I will use screen.
The alternative is of course to use the different virtual consoles [ctrl alt F1 -F6], but you can do a lot of other nice things with screen so it is recommended.
#yum install screen
$screen

Now let’s change the default ssh port. In /etc/ssh/sshd_config find ‘#port 22’ and remove the ‘#’ and set the port to 50022:
port 50022

Find PermitRootLogin and set it to no
PermitRootLogin no

Save and exit.
You will need to restart the ssh server for the changes to take effect:
#systemctl restart sshd

Go a head and try it if you want:
$ssh admin@localhost
ssh: connection to host localhost port 22: Connection refused
$ssh root@localhost -p 50222
Permission denied, please try again
$ssh admin@localhost -p 50222

Step 2 – setting up network


We will open up one more guest. This guest needs a desktop environment since we will use a web browser. I will use a Xubuntu guest. Let’s call the CentOS guest C and the Xubuntu guest X.
First add both guest C and X to a local network like we did previous. Let C still have it’s NAT adapter, but remove the cable.

webserver_networki
Get the ip of C:
$ip addr
192.168.3.2

Make sure we can ssh from X to C:
$ssh admin@192.168.3.2 -p 50022
ssh: connect to host 192.168.3.2 port 50022: No route to host

Strange, let’s see if we can ping C:
$ping 192.168.3.2
4 packets transmitted, 4 received, 0% packet loss, time 3005ms

It seem as we can reach the host. The port seem to be closed. Something is blocking it, most likely a firewall.

Step 3 – iptables


Many Linux distros comes with a firewall/IP filter called iptables*. It’s a really complex and advanced module and I will only cover some basics in this tutorial. If you got some time check out this awesome tutorial.

Make sure your NAT adapter is unplugged for guest C. Now let’s turn of iptables:
#iptables -P INPUT ACCEPT
#iptables -F

The first line allows all input (so make sure internet adapter is unplugged!). The second line flushes the table.

Retry to ssh from X to C.
It works. Now we need to add some real rules to iptables.
We want to allow ssh connections on port 50022, and since we will set up a web server we will also open port 80.

#iptables -P INPUT DROP
This line adds a policy to drop all incoming traffic.
We still want to accept http and ssh so we need to add some exceptions:
#iptables -A INPUT -p tcp –dport 50022 -j ACCEPT
#iptables -A INPUT -p tcp –dport 80 -j ACCEPT
Both this lines makes the firewall accept port 50022 and 80 for tcp traffic.

#iptables -P FORWARD DROP
Drop all forwarding
#iptables -P OUTPUT DROP
Allow all outgoing traffic

Lastly we will allow incoming traffic for already established or related traffic.
#iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

* CentOS 7 uses firewallD – but I still had to open the port in iptables.

Step 4 -apache


Connect your adapter to internet again and install apache:
#yum install httpd
Then start the service:
#systemctl start httpd.service

Now from guest X open the web browser and enter the ip address of C.
You should see the default test page.

connected

 

Conclusion


It’s quite easy to create a web server on a virtual machine. Usually when you use a virtual machine for hosting you use the VM in a headless state and connect to it through ssh – there’s no point of having a GUI at all. This requires static IP and that the host forwards the traffic to the guest though. We will have a look in upcoming posts of some alternatives to connect to your virtual web server from the outside.

Virtuabox – Networks

VM makes it possible to have a network of different guests, an useful tool when studding network or server development. In this guide I show how to set up a network of guests.

Requirements:
Basic networking

Time:
Around 30 min

Table of content

Introduction


Sooner or later you want to connect to one of your guest OS. But they don’t have a physical network adapter and they are hidden behind your host. It’s actually not as tricky as it seem, virtualbox comes with its own DHCP server that you can configure and you can select different types of network adapters.

Because we will use multiple guests I will refer them to guest A and guest B. For this guide both will be Xubuntu, like we set up previously, but any kind of Ubuntu clone should work.

Step 0 – Setup ssh


To test that we can reach other machines we will connect with ssh. Start guest A.
The ssh client is already installed, but we will have to install the ssh server that makes it possible to connect to your machine.
#apt-get install openssh-server

The configuration for sshd is in /etc/ssh/sshd_config
We will edit it, so install your favorite editor – I prefer vim:
#apt-get install vim
Then:
#vim /etc/ssh/sshd_config

There’s a lot of settings but what I normally do right after installation is to set “PermitRootLogin no” and change the port. Both these changes are for security reasons. We will change the port in another post so for now just change PermitRootLogin.

Start it by*:
#service ssh start
To check the status of ssh on Xubuntu:
#service ssh status
ssh start/running, process 5283

Last thing to do is to test it by connecting to our self:
$ssh <username>@localhost

 

ssh_first


* On most modern Linux system you should use systemctl, however it is not adapted by xubuntu 14. If your distro use systemctl try:
#systemctl start sshd

Step 1 – Setup the Network


We need to know the IP of guest A:
$ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:c1:53:16 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0

That means the guest has IP 10.0.2.15 on eth0. The device id “eth0” will be important later when we use multiple network interfaces.

Start guest B and keep A running. Since we won’t connect to guest B only from it, it’s not necessary to install the ssh-server.
Try to connect to guest A from B:
$ssh <username>@10.0.2.15
ssh: connect to host 10.0.2.15 port 22: Connection refused

Refused, check what the ip for guest B is:
$ip addr
inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0

Hm, that’s not good ,they have same IP.

The guests are also hidden from each other behind the host, so they’re given the same IP. The host doesn’t automatically route the traffic. What we need is a LAN where the guest can reach each other.

Step 2 – Setting dhcpserver


We need a DHCP Server. A DHCP Server is responsible for giving devices on a network a valid IP address, normally your wireless router does this for you at home.

Open terminal in your host:
(If you use windows go to your virtalbox installation folder, default is ‘program files\oracle\virtualbox’)

The parameters you need to set is the following:
–netname: This must be the same as the network name for adapter 3. I choose “inet-1”
ip: This is the IP to your DHCP server, Don’t choose the same as for your host’s network. My host has 192.168.0.1 so I picked 192.168.3.1 (I like to use the same digit as my adapter for the second last number).
–netmask: Depending on how big network you want, but should probably be 255.255.255.0
–lowerip: Lowest IP a connected machine can have. I choose 192.168.3.2 (avoid ending with 0 or 1).
–upperip: Highest IP a connection can have. I picked 192.168.3.254 (avoid ending with 255)
–enable: This flag must be set or the DHCP Server won’t be used.


See the manual
for further information about dhcpserver.

My command look like this:
>VBoxManage dhcpserver add –netname inet-1 -ip 192.168.3.1 –netmask 255.255.255.0 –lowerip 192.168.1.2 –upperip 192.168.3.254 –enable

Turn off the machines so that you can modify them. Select Guest A in virtualbox and go to Network. Adapter 1 shows attached to NAT, network address translation. In other words the adapter just translate the traffics destination to the guest. Disable it and select adapter 3.
Change the attach to internal network to “inet-1”.
Do the same for guest B.

inet
Restart the guests. Try ip addr on guest A and B:
A:
inet 192.168.3.3/24 brd 192.168.3.255 scope global eth0
B:
inet 192.168.3.2/24 brd 192.168.3.255 scope global eth0

Looks good, let’s try ssh from B to A now:
$ssh <username>@192.168.3.3
ssh_success

Step 3 – Where’s my internet?


If you start a web browser you notice that you cannot connect to internet anymore. Since your adapter is only connected to LAN you can’t reach to the outside world anymore. The easiest fix for this is to add adapter 1 again.
However if you do ip addr you will notice that you now have two network interface with their own IP:
inet 10.0.2.15/24 brd 10.0.2.255 scope global eth0
inet 192.168.3.3/24 brd 192.168.3.255 scope global eth1

What we have done here is to plug one cable to the internet outlet and we one to our LAN router. Some times this is what we want, but it is not a very common setup. What we would like to do is to have our router connected to internet and then translating the addresses for all the machines in the LAN. This can be done by NAT Network.

Open a console on your host.

The parameters you need to set is the following:
–netname: the name of the NAT network. I choose nnet-1
–network: the ip of this network. This will also be the range of IP that the dhcp will give to connected machines. I choose 192.168.2.0/24
–enable: this flag must be set to use the network
–dhcp: If you want dhcp set it to “on”

See the manual for more information about natnetwork

My command line look like this:

>VBoxManage natnetwork add –netname nnet-1 –network 192.168.3.0/24 –enable –dhcp on
Disable adapter 1 and 3. Enable Adapter 2 and select attached to NAT Network and select the network we just created.

Start your guests again and make sure you can use internet.
Then try to connect from B to A again by ssh.

It works!

Conclusion


Now you can set up a network with several guests. It is much cheaper than buying several physical computers and requires less space and power. But what kind of Network should you choose? If you are playing around with a web server it is more secure to use the LAN solution, but it is also troublesome if you want to install software.
If you only want to use internet some times you could do the LAN solution with a NAT adapter. You can at any time uncheck the “Cable connected” and you will go offline.

Virtualbox – Android Development Setup

In this guide I will show how to set up a Linux guest OS for Android development.

Requirements:
Basic Linux usage.
Basic Android Studio usage.
Virtualbox installed.

Toys/Equipment:
Android Device – If you want to learn how to run your app on a device and not emulator. I’m using a Sony z1.

Time:
Around 30 min.

Table of content

Introduction


Android is based on Linux and most tools for developing is made for Linux. I used to develop using Windows but it was never as smooth as for Linux. Sometimes my work has forced me to use a Windows machine and then the first thing I do is to install a linux VM. I really hate to use Cygwin – why try imitate Linux when you can have the real thing?
Now using virtualbox for Android isn’t totally pain free since you need to use adb through USB, something that can be tricky to set up correctly, especially if your host is Linux.

Step 0 – Xubuntu


When I first started to use Ubuntu I loved it. It was not full of things I didn’t need as in Windows and it was quite user friendly. However, after a while I started to feel it was bloated and I never liked Unity. Unity throws big icons in my face and more than a few times it is incompatible and you have to do a lot of research to solve the problems.
If you are going to use your VM for Android development I instead recommend Xubuntu. It is very similar to Ubuntu, but it is stripped down and the desktop environment xfce is much more in my taste.

In this guide I will be using Xubuntu, but it should be fine to use any Ubuntu clone.
Download and install Xubuntu here:
http://xubuntu.org/

In this guide I’m using Xubuntu 14.04 64-bit. First thing to do is to is to update the guest OS. Note that for this guide ‘$’ means normal shell command, while ‘#’ means executed as super user.
#apt-get update
#apt-get upgrade

Step 1 – Why is my screen so small?!


This is probably everyone’s first question when they have installed a Linux guest.
Pressing [right ctrl + f] enter full screen mode, but still the working area is 800×600. You need to install the guest additions to get a higher resolution. The good news is that for Ubuntu it is easy (while it is somewhat more demanding for other distros such as CentOS).

Install the guest addition for x11 and reboot:
#apt-get install virtualbox-guest-x11
#reboot

Ah, that’s better!

Step 2 – Android Studio


Android Studio requires Java, and of course if you are going to develop for android you need Java 7+. Xubuntu doesn’t come with java so you have to install:
#add-apt-repository ppa:webupd8team/java
#apt-get update
#apt-get install oracle-java8-installer

java_alternatives

Check the configuration with:
#update-alternatives–config java

Download Android Studio:
https://developer.android.com/sdk/index.html#Other

Make sure you choose “All Android Packages” and not “SDK tools only”. In this guide I have used android-studio-ide-141.1980579-linux.zip.
Extract the archive. I extracted it to ~/android/android-studio

If you have 64-bit version of your guest, then you will have to install some 32-but libraries to be able to run Android Studio.
#apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6

Start android studio:
$android/android-studio/bin/studio.sh

I found Darcula them doesn’t look good in Xubuntu so I choose the default IntelliJ.
You will have to download the SDK. At some of my positions we had our own SDK but I haven’t found a way to skip this step. I installed it to ~/android/sdk

Step 3 – Android Project


Create a new Android Project with a blank activity. Connect your Android Device with USB. Run the app [shift F10]
The Choose Device dialog is empty.

Check the status of the USB on the guest:
$lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 80ee:0021 VirtualBox USB Tablet
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

Hmm… there’s no USB device connected. Try to run the same command on your host if you run Linux, or check “This PC” if you use windows. The USB is visible at the host right?

To allow the guest to detect USB you need to add it to the VM USB Device Filters.
Open virtualbox and click USB and then on the USB with a green plus sign. You should see your device in this list. Select it and make sure it is checked and then press OK.


Only one OS (host or guest) can control an USB device at the same time. This is important to remember when you troubleshot your USB. The Device Filter control if the guest OS should use the USB or the host.
Disconnect the USB and reconnect it and check for devices on your guest.
$lsusb
Bus 001 Device 004: ID 0fce:d107 Sony Ericsson Mobile Communications AB

That’s better!
Don’t forget to check the “USB debugging” in developers option of the Android Device.

Step 4 -Troubleshooting


“Unable to run mksdcard SDK tool” when installing the SDK:
You are missing 32-bit libraries:
#apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6

I use Linux and I can’t see my USB device:
You need to add yourself to vboxusers group, or you wont have permission to access the USB devices.
#adduser $USER vboxusers

https://help.ubuntu.com/community/VirtualBox/USB

I can find my device but not use it in Android studio:
Make sure USB debugging is on on the android device.

adb devices shows ?????? as id:
adb does not have the correct premissions. Try to restart it as super user:
$adb kill-server
#adb start-server

Conclusion


Setting up an Android development environment on a guest is quite same as on a host. The only thing you need to consider is the USB support. Your key tools when trouble shooting  is lsusb and adb.  adb devices gives you information about the android device, while lsusb gives you information about USB devices.
I prefer Xubuntu, but if you want a really easy installation I recommend Linux mint http://www.linuxmint.com/download.php. It automatically installs the guest drivers (and cinnamon looks damn good).  Cooperates should really consider using mint instead of Ubuntu.