Notes RE: Smartos

I'm trying to figure out how to run SmartOS on kimsufi; ideally redirecting ports towards KVM instances. Don't have much now, but here's some links I'm getting:

SmartOS wiki:
https://wiki.smartos.org/display/DOC/Home

IRC log discussing setting up LX on Smartos:
http://echelog.com/logs/browse/smartos/1429567200

Single IP tips for SmartOS:
https://docu.blackdot.be/snipets/solaris/smartos-nat

Tips for using proxmox

I just got a server and one of the options was to run proxmox on it, so I chose that.

I found this guide on low-end talk which had some great tips on it. Most importantly was the interfaces entry and network info, which I'm gonna re-print here in the event that link dies.

Put this in /etc/networks/interfaces to create a working bridge entry:

auto vmbr2
iface vmbr2 inet static
address 192.168.0.254
netmask 255.255.255.0
bridge_ports none
bridge_stp off
bridge_fd 0
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
post-up iptables -t nat -A POSTROUTING -s '192.168.0.0/24' -o vmbr0 -j MASQUERADE
post-down iptables -t nat -D POSTROUTING -s '192.168.0.0/24' -o vmbr0 -j MASQUERADE


Then restart the server.

Use these settings when configuring VM networks:
ip : 192.168.0.1 to 192.168.0.250
subnet mask 255.255.255.0
gateway : 192.168.0.254 (ip vmbr2 on the host)
DNS: your server's DNS or google's (8.8.8.8, 8.8.4.4)

It's weird to delete proxmox virtual machines, what I do is:
ssh as root

1)delete the configuration file under /etc/pve/qemu-server
2)delete the disk image file under /var/lib/vz/images/

The web interface will refresh and the vm will be gone, as an added bonus you'll be able to re-use the node number for a new vm.

Getting a higher resolution qemu on openindiana Oct 2015

I found something interesting on the mailing list today. Apparently openindiana has some sort of support for virtual consoles -only on certain computers, though; and not for qemu, as I found out the hard way.

First, here's the wiki entry; http://wiki.openindiana.org/oi/4.6+Remote+Console

As I said, this won't work on kvm. But by trying to get it to work I was able to finally break free of the low-res hell that I was stuck in with qemu/vga -std. On Krexel's essay "qemu: using cirrus considered harmful" I found an interesting tidbit which will prove very useful. You can set qemu's video ram from the command line. The way I've done it with my hipster box is to edit "-std vga" to "-vga std -global VGA.vgamem_mb=256 " which is admittedly overkill, but it seemed to help.


Then I simply pasted the xorg.conf I found on this page:
https://help.ubuntu.com/community/KVM/FAQ --deleting the mouse section (to prevent a potential conflict with the illumos mouse driver).

Going back to the wiki link I posted earlier, I used and ssh session to turn gdm off with this command: svcadm disable svc:/application/graphical-login/gdm:default ...followed by
svcadm enable svc:/application/graphical-login/gdm:default to turn it back on to test my changes.

There's a down-side, though. I edited /etc/X11/xorg.conf and set my screen resolution to 1280x1020 and it seems to play hell with both Qemu's sdl and gvncviewer; if I solve that problem, I'll write a follow-up or edit this post.

[EDIT] ...and here's that update! I'm finding that I can simply add "-full-screen" to the commandline, and/or do ctrl-alt-f to go fullscreen. Starting Hipster Oct 2015 in graphical mode gets weird, but switching out of and then back into full screen seems to fix that, at least so far.

Toying with omnios

Just some cliff notes for future use:

1)setting up networking:
(as root):
touch /etc/dhcp.e1000g0
touch /etc/hostname.e1000g0

2)edit /etc/hosts, add a line for your host name (replace 'omni01' with whatever name you want):
127.0.0.1       omni01 omni01

3)add a user (there has to be a better way to do this!):
useradd -s `which bash` USERNAME
mkdir /export/home/USERNAME
chown USERNAME.staff /export/home/USERNAME

4)edit /etc/passwd and change USERNAME's home to /export/home/USERNAME (careful with that axe!)

...that's all, so far.

quick note, illumos rebuild, Sept 19, 2015

As usual, this was on kvm;
the qemu command prompt;

 qemu-system-x86_64 -enable-kvm -cpu host,-x2apic -m 2048 -smp 3 -net nic,model=e1000 -net user -redir tcp:5501::22 -drive file=hip/hip_12Sept2015.img,index=0,media=disk,format=raw -vga std -vnc :0

/usr/bin/time results from inside of the vm:
rliegh@virtualindiana:/code/illumos-gate$ time ./nightly.sh illumos.sh

real    61m10.830s
user    121m47.677s
sys     33m45.621s
rliegh@virtualindiana:/code/illumos-gate$

A few links

I'm too lazy to figure out how to add a blogroll with the theme I've chosen, so instead I'm going to post the blogs and resources I find and like here:


gunkies -lots of computing history and tutorials;
http://gunkies.org/wiki/Category:Operating_Systems
http://gunkies.org/wiki/Category:QEMU_Tutorials

neoseed's excellent blog and resources -a bit of everything;
http://virtuallyfun.superglobalmegacorp.com/

A great blog by one of the developers of OpenBios; handy for doing things on qemu-sparc;
http://tyom.blogspot.com/

Firstworks systems; he explores a lot of 'nixes, mostly using vmware it seems:
http://systems.firstworks.com



I'm on a qemu kick, so here's a few pages about that!
https://en.wikibooks.org/wiki/QEMU
https://wiki.archlinux.org/index.php/QEMU
https://wiki.gentoo.org/wiki/QEMU
http://wiki.qemu.org/Documentation/Networking#VDE

Useful exchange on stackexchange about sparc64 on qemu;
http://unix.stackexchange.com/questions/199827/booting-solaris-10-or-11-for-sparc-in-qemu-system-sparc64

Rodovan Brezula's experiences with qemu and sparc;
http://brezular.com/2012/02/12/qemu-installation-on-fedora-linux/
http://brezular.com/2012/02/17/installation-solaris-2-6-sparc-on-qemu-part2-solaris-installation/
http://brezular.com/2012/04/08/installation-solaris-sparc-2-6-sunos-5-6-on-qemu-part3-iou2net-pl-installation/


On a semi-related note, selecting the cpu for qemu-system-sparc64 is tricky, at least I found it to be. Here's how you do it.
Select your target cpu from the list, for instance;

Sparc Sun UltraSparc IV+ IU 003e001922000000 FPU 00000000 MMU 00000000 NWINS 8 +cmt 

for that one, you would ignore the leading "Sparc" and eveything after "IV+", so you end up with
-cpu "Sun UltraSparc IV+"

Building NetBSD 7.0RC3 on KVM

Today I tested out building NetBSD 7 on a NetBSD 6 KVM virtual machine.

Here's the command line for starting the vm:
===
qemu-system-x86_64 -smp 4 -cpu host -enable-kvm -m 2048 -vga std -vnc :0 -drive file=$PWD/nbsd/wd01.img,index=0,media=disk,format=raw -net nic,model=e1000 -net user -redir tcp:5550::22 &
===
here's the post-build summary from build.sh:
===
         build.sh command:    ./build.sh -O /usr/obj -U -j 4 tools kernel=RND0 modules distribution sets
         build.sh started:    Mon Sep 14 16:58:05 AKDT 2015
         NetBSD version:      7.0_RC3
         MACHINE:             amd64
         MACHINE_ARCH:        x86_64
         Build platform:      NetBSD 6.1.5 amd64
         HOST_SH:             /bin/sh
         MAKECONF file:       /etc/mk.conf (File not found)
         TOOLDIR path:        /usr/obj/tooldir.NetBSD-6.1.5-amd64
         DESTDIR path:        /usr/obj/destdir.amd64
         RELEASEDIR path:     /usr/obj/releasedir
         Updated makewrapper: /usr/obj/tooldir.NetBSD-6.1.5-amd64/bin/nbmake-amd64
         Tools built to /usr/obj/tooldir.NetBSD-6.1.5-amd64
         Building kernel:     RND0
         Build directory:     /usr/obj/sys/arch/amd64/compile/RND0
         Kernels built from RND0:
          /usr/obj/sys/arch/amd64/compile/RND0/netbsd
         Building kernel modules for NetBSD/amd64 7.0_RC3
         Successful build of kernel modules for NetBSD/amd64 7.0_RC3
         Successful make distribution
         Building sets from pre-populated /usr/obj/destdir.amd64
         Built sets to /usr/obj/releasedir/amd64/binary/sets
         build.sh ended:      Mon Sep 14 18:08:47 AKDT 2015
===
Testing and installing it now -90 minutes isn't too shabby for a VM build. The host machine barely felt it, btw; no real noticeable performance hit.
===
I blitzed the install by following the instructions on the guide, but that doesn't matter that much -I found out what I wanted to know.

Fun with KVM and illumos

According to the "building illumos" page, KVM is better for building illumos than Virtualbox. Right now I cannot run on bare metal so some sort of virtualization platform is required.

However, none of the illumos-based options worked for me on kvm; the network adapter fails. Last night I got lucky in my googling and I found out that qemu 2.0 made changes that the illumos network drivers don't work with.

It's easy to work around once you know about it, actually. In order to get it to work you simply tell it not to use the x2apic feature of qemu's cpu.

Like this: "-cpu host,-x2apic "
That requires kvm, but you could easily use "-cpu qemu64,-x2apic" which wouldn't. For performance reasons, you'll want to enable kvm if at all possible.

Other neat features I've unlocked in the last few days:

Shutting off raw warnings by using "-drive" instead of "-hda":

-drive file=FILENAME,index=0,media=disk,format=raw

Using qemu's built-in vnc server:
 -vnc :0
...I the connect with gvncviewer 127.0.0.1:0

Enable qemu's built-in smb server to transfer files:
-smb YOUR/PATH/HERE

Use network redirection to allow myself to ssh into the VM:
-redir tcp:5555::22

Once I figured out all of those, it was time to set up a Hipster virtual machine; I used an iso i had grabbed from the OI Hipster download site and installed it.

Nothing really remarkable here, though I decided that I didn't need the overhead of running X and GNOME in the VM so I ran sudo -s and then from the prompt disabled gdm:

#svcadm disable svc:/application/graphical-login/gdm

I also created a small shell file to set up my samba share inside the vm:

cat bin/mntq.sh
#!/bin/bash
echo "type user password here:"
sudo svcadm enable -r network/smb/client
mount -F smbfs //10.0.2.4/Qemu $HOME/qemu

From there I simply tabbed between firefox and my mate terminal (with ssh to the vm running) and cut and pasted commands from the illumos build instructions.

It took a while, a bit longer than it would have on bare metal but eventually I got what I was after:

./nightly.sh illumos.sh


real    101m46.436s
user    154m30.348s
sys     122m32.661s

Host machine:
i5 quad core,
Linux MINT 17/x64
12 gigs of ram

Guest machine:
-enable_kvm
-cpu host,-x2apic
-smp 4
2 gigs of ram


I'm hoping there's more tunage I can do; particularly with the disk and network; but this is all pretty good progress for just one day!