USB Auto mapping to Windows VM under KVM does not work

Let me first say, that it does work for Linux guest. It doesn’t work on Windows guest because there is a know bug (/issue) with the default hardware layout – made of i440FX BIOS. VirtManager would not allow us to replace the settings, so we need to create the VM ourselves using XML. You can export your XML settings (of an existing VM) using the command

virsh dumpxml  > /tmp/VM_NAME.xml

There are relevant fields there which you might want to save for later, like MAC addresses, network settings, and so on.

You can use this XML file to build your VM anew. Note that you will want to modify the network settings, the name and the UUID. Also – you will need a newer QEMU command (through the package qemu-system-x86), you can find in the Centos updates repository, . It has been providing me with /usr/bin/qemu-system-x86_64 command, which I am using, instead of the default qemu command used by default by VirtManager.

My Windows VM XML file (as a reference you can copy and use) is provided below. Major modifications are required to the hardware settings of the Windows VM – moving from PCI to PCIE, changing from IDE to SATA or VirtIO – and the provided XML gives a good reference of how this file should look like. This was taken from a machine tested to allow USB hot-add/remove via the method provided in my previous post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
<domain type='kvm' id='1'>
  <name>Windows</name>                             <!-- Change the name to match your settings -->
  <uuid>9a10dc43-5c39-411d-8dc9-f6c6b849d212</uuid> <!-- Make sure you change the UUID. Dont have duplicates -->
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-2.0'>hvm</type> <!-- This is the key part - using different machine type -->
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator> <!-- This is important! -->
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/images/Windows-test1.qcow2'/>  <!-- Point to the right file. It can be raw or qcow2, so change the line above accordingly -->
      <backingStore/>
      <target dev='sda' bus='sata'/> <!-- I have used SATA and not VirtIO, because the later requires drivers on Windows. I will need to handle that in the future, but for our example, it should work -->
      <alias name='sata0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <backingStore/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <boot order='1'/>
      <alias name='sata0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'>
      <model name='i82801b11-bridge'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <model name='pci-bridge'/>
      <target chassisNr='2'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='3' port='0x10'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x06' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <interface type='bridge'>
      <mac address='11:12:14:12:a1:03'/>  <!-- Change the MAC address! -->
      <source bridge='bridge'/>
      <model type='rtl8139'/>    <!-- Also wanted VirtIO, but needs drivers -->
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/10'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/10'>
      <source path='/dev/pts/10'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='spice' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich6'>
      <alias name='sound0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir0'/>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir1'/>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <stats period='5'/>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='none' model='none'/>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+107:+107</label>
    <imagelabel>+107:+107</imagelabel>
  </seclabel>
</domain>

Now just import the VM using

virsh define /tmp/NEW_MACHINE.xml

and your changes are available via VirtManager, where you can edit (some of them).

Auto mapping USB Disk on Key to KVM VM using libvirt and udev

I was required to auto map a USB DoK to a KVM VM (specific VM, mind you!), as a result of connecting this device to the host. I’ve looked it up on the Internet, and the closest I could get there was this link. It was almost a complete solution, but it had a few bugs, so I will re-describe the whole process, with the fixes I’ve added to the process and udev rules file. While this guide is rather old, it did solve my requirement, which was to map a specific set of devices (“known USB devices”) to the VM, and not any and every USB device (or even – USB DoK) connected to the system.

In my example, I’ve used SanDisk Corp. Ultra Fit, which its USB identifier is 0781:5583, as can be seen using ‘lsusb’ command:

[[email protected] ~]# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 020: ID 0781:5583 SanDisk Corp. Ultra Fit

My VM is called “centos7.0” in this example. I am using integrated KVM+QEMU+LIBVIRT on a generic CentOS 7.5 system.

Preparation

You will need to prepare two files:

  • USB definitions file (for easier config of libvirt)
  • UDEV rules file (which will be triggered by add/remove operation, and will call the USB definitions file)
USB Definitions file

I’ve placed it in /opt/autousb/hostdev-0781:5583.xml , and it holds the following (mind the USB device identifiers!)

1
2
3
4
5
6
<hostdev mode='subsystem' type='usb'>
  <source>
    <vendor id='0x0781'/>
    <product id='0x5583'/>
  </source>
</hostdev>

I’ve created a file /etc/udev/rules.d/90-libvirt-usb.rules with the content below. Note that the device identifiers are there, but in the “remove” section they appear differently. Remove leading zero(s) and change the string. This is caused because on removal, the device does not report all its properties to the OS. Also – you cannot connect more than three (3) such devices to a VM, so when you fail to detach three devices (following a consecutive insert/remove operations, for example), you will not be able to attach a fourth time.

ACTION=="add", \
    SUBSYSTEM=="usb", \
    ENV{ID_VENDOR_ID}=="0781", \
    ENV{ID_MODEL_ID}=="5583", \
    RUN+="/usr/bin/virsh attach-device centos7.0 /opt/autousb/hostdev-0781:5583.xml"
ACTION=="remove", \
    SUBSYSTEM=="usb", \
    ENV{PRODUCT}=="781/5583/100", \
    RUN+="/usr/bin/virsh detach-device centos7.0 /opt/autousb/config/hostdev-0781:5583.xml"

Now, all that’s left to do is to reload udev using the following command:

udevadm trigger

To monitor the system behaviour, run either of these commands:

udevadm monitor --property --udev 

or

udevadm monitor --environment --udev 

Oracle 12.2 Grid Infrastructure installation tips

There are many sites explaining how to install Oracle GI 12.2, however, there are some special tricks which can simplify GI installation.

For once, when installing GI and then installing the huge PatchSet (which is usually around 1.4GB in size) – it takes time. A lot of time. A simple but not very well documented trick is to run the installer with a specific flag, pointing to the extracted PSU. You can obtain the PSU from Oracle document ID 2118136.2 (Oracle support plan required).

After extracting the contents of the oracle grid home to the destination directory, and after extracting the contents of the PSU to a known (other) location, run from the oracle grid home directory the following command:

./gridSetup.sh -applyPSU /path/to/PSU

(case sensitive). You will need a working $DISPLAY because following the patch apply phase, an installation window will pop up.

That said, make sure you remove (rpm -e –nodeps stix-fonts) if you are on RHEL/OEL/Centos version newer than 7.4. These fonts will prevent the GUI installer from starting (java would crash) and will cause great frustration. You can later on restore this package if you feel the urge.

Additional trick I’ve seen, but yet to try, is how to run the GI installer unattended. This can be done like this:

./gridSetup.sh -silent -responseFile /path/to/response/file.rsp
< run root.sh as directed >
./gridSetup.sh -executeConfigTools -all -silent -responseFile /path/to/response/file.rsp

Hope this helps.

The StartTLS replacement for the old telnet to SMTP server on port 25

When you need to troubleshoot SMTP issues, it is a known fact that a simple telnet to port 25 of the SMTP server in question would get you far. It will get you to see the problems.

When connecting to Office365 (outlook.com) to relay mail, and you want to check how things work, you can use openssl to wrap in StartTLS your old telnet connection by running this:

openssl s_client -starttls smtp -crlf -connect smtp.office365.com:587

From there, you can run your plain old “ehlo user” and all these commands like you are used to.

Just a small note about authentication: if you are facing SMTP which requires authentication, there are few methods you can use. Let’s assume your user is ‘[email protected]’ and your password is ‘password’.

If you are allowed to use the PLAIN method, you need to generate the login/password string into base64, like this:

perl -MMIME::Base64 -e ‘print encode_base64(“\000user\@domain.com\000password”)’

You could use shell with base64 command to perform the convertion:

echo -ne ‘\[email protected]\0password’ | base64

The result would be a string similar in shape to this: AHVzZXJAZG9tYWluLmNvbQBwYXNzd29yZA==

Then enter the SMTP server at the right prompt:

AUTH PLAIN AHVzZXJAZG9tYWluLmNvbQBwYXNzd29yZA==

If you are allowed to use the LOGIN method, you need to generate base64 string for your user and your password separately, like this:

perl -MMIME::Base64 -e ‘print encode_base64(“user\@domain.com”)’

or

echo -ne ‘[email protected]’ | base64

The result is dXNlckBkb21haW4uY29t

Same goes for the password field. Choose which of the next two lines you wish to use:

perl -MMIME::Base64 -e ‘print encode_base64(“password”)’

echo -ne ‘password’ | base64

The result is cGFzc3dvcmQ=

Now, for the prompt, we will run:

AUTH LOGIN

We’ll get the base64 query for username, so we just type/paste the user base64, and press on Enter. Then we’ll get the base64 prompt for password, so we will type/paste the password base64, and press Enter.

That’s all.

Asus wireless router and VLAN tagging

The idea in general is to have multiple wireless networks at home – one for the house residents, the other for visitors. The home network should have full access to everything, while the guest network should be able to reach the Internet, but nothing else.

I have Asus RT-AC87U, which is a fine router, but does not show these capabilities in its web GUI. I had flushed it with a derived firmware called AsusWRT-Merlin which added the ability to insert custom scripts.

I’ve had to research a bit, until I got something working. For future tinkering, and for any who requires it, I will add my scripts here.

First – in the web interface, enable guest network and, under Administration->System enable JFFS custom scripts.

Then, connect via SSH to the router, and place a script called /jffs/scripts/services-start containing:

#!/bin/sh
touch /tmp/000brstarted
PATH=”/sbin:/usr/sbin:/bin:/usr/bin:${PATH}”
robocfg vlan 100 ports “1t 2t 3t 4t 5t 8t”
vconfig add eth0 100
ifconfig vlan100 up
brctl addbr br1
brctl addif br1 vlan100
brctl delif br0 wl0.1
brctl addif br1 wl0.1
ifconfig br1 192.168.230.254 netmask 255.255.255.0 up
nvram set lan_ifnames=”vlan1 eth1″
nvram set lan_ifname=”br0″
nvram set lan1_ifnames=”vlan100 wl0.1″
nvram set lan1_ifname=”br1″
nvram set lan1_ipaddr=192.168.230.254
nvram commit
killall eapd
eapd

Run chmod +x /jffs/scripts/services-start so that it will work correctly.

This script will configure VLAN100 on all ports (including the internal ones 5 and 8), as VLAN tags (meaning – not access). Then it will add the VLAN to eth0 – which is the host interface for the external switch ports (eth1 is for the Wireless ports), bring it up, and create a bridge consisting of vlan100 and the additional wireless sub-interface wl0.1 (which is the guest interface). I did not bother setting up 5GHz guest network, so I didn’t have an additional wl1.1 sub-interface. If you configure a 5GHz guest network, you will need to add it to the bridge device. Then I’ve given the bridge interface an IP address so I could test it from my router, and setup nvram to hold these settings. Unfortunately, these settings must be defined each boot, and they are not kept without the script.

Maybe on my next post I will describe my switch network layout and settings. On a future post, I might even describe how to transfer VLANs to a VM running under KVM, and maybe even explain my router settings, so that eventually the readers (other than myself, of course) could reproduce this setup at their homes.