SPARC Logical Domains: Alternate Service Domains Part 2
In Part One of this series, we went through the initial configuration of our Logical Domain hypervisor and took some time to explain the process of mapping out the PCI Root Complexes, so that we would be able to effectively split them between the primary and an alternate domain.
In Part Two (this article) we are going to take that information and split out our PCI Root Complexes and configure and install an alternate domain. At the end of this article, you will be able to reboot the primary domain without impacting the operation of the alternate domain.
In Part Three we will be creating redundant virtual services as well as some guests that will use the redundant services that we created, and will go through some testing to see the capabilities of this architecture.
Remove PCI Roots From Primary
The changes that we need to make will require that we put LDM into dynamic reconfiguration mode, which will require a reboot to implement the changes. This mode also prevents further changes to other domains.
# ldm start-reconf primary<br />
Initiating a delayed reconfiguration operation on the primary domain.<br />
All configuration changes for other domains are disabled until the primary<br />
domain reboots, at which time the new configuration for the primary domain<br />
will also take effect.
Now we remove the unneeded PCI Roots from the primary domain, this will allow us to assign them to the alternate domain.
# ldm remove-io pci_1 primary<br />
——————————————————————————<br />
Notice: The primary domain is in the process of a delayed reconfiguration.<br />
Any changes made to the primary domain will only take effect after it reboots.<br />
——————————————————————————<br />
# ldm remove-io pci_3 primary<br />
——————————————————————————<br />
Notice: The primary domain is in the process of a delayed reconfiguration.<br />
Any changes made to the primary domain will only take effect after it reboots.<br />
——————————————————————————
Lets save our configuration.
# ldm add-config reduced-io
Now a reboot to make the configuration active.
# reboot
When it comes back up we should see the PCI Roots unassigned.
Create Alternate Domain
Now we can create our alternate domain and assign it some resources.
# ldm add-domain alternate<br />
# ldm set-vcpu 16 alternate<br />
# ldm set-memory 16G alternate
We have set this with 2 cores and 16GB of RAM. Your sizing will depend on your use case.
Add PCI Devices to Alternate Domain
We are assigning pci_1 and pci_3 to the alternate domain, this will have direct access to two of the on-board NICs, two of the disks, and half of the PCI slots. It also will inherit the CDROM as well as the USB controller.
Also really quick I just wanted to point this out quickly. The disks are not split evenly, pci_0 has 4 disks, while pci_3 only has two. So that said if your configuration included 6 disks then I would recommend using the third and fourth in the primary as non-redundant storage pool, perhaps to be used to stage firmware and such for patching. But the bottom line is that you need to purchase the hardware with 4 drives minimum.
# ldm add-io pci_1 alternate<br />
# ldm add-io pci_3 alternate
Here we have NICs and disks on our alternate domain, now we just need something to boot from and we can get the install going.
Lets save our config before moving on.
# ldm add-config alternate-domain
With the config saved we can move on to the next steps.
Install Alternate Domain
We should still have our CD in from the install of the primary domain. After switching the PCI Root Complexes the CD drive will be presented to the alternate domain (as it is attached to pci_3).
First thing to do is bind our domain.
# ldm bind alternate
Then we need to start the domain.
# ldm start alternate
We need to do is determine what port telnet is listening on for this particular domain. In our case we can see it is 5000.
# ldm ls<br />
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME<br />
primary active -n-cv- UART 16 16G 0.2% 0.2% 17h 32m<br />
alternate active -n–v- 5000 16 16G 0.0% 0.0% 17h 45m
When using these various consoles you always need to be attentive to the escape sequence, which in the case of telnet it is ^], which is “CTRL” + “]” once we have determined where we can telnet to, then we can start the connection. Also important to note. You will see ::1: Connection refused. This is because we are connecting to localhost, if you don’t want to see that error connect to 127.0.0.1 (which is the IPv4 local address).
# telnet localhost 5000<br />
Trying ::1…<br />
telnet: connect to address ::1: Connection refused<br />
Trying 127.0.0.1…<br />
Connected to AK00176306.<br />
Escape character is ‘^]’.</p>
<p>Connecting to console "alternate" in group "alternate" ….<br />
Press ~? for control options ..</p>
<p>telnet> quit<br />
Connection to AK00176306 closed.
I will let you go through the install on your own, but I am assuming that you know how to install the OS itself.
Now let’s save our config, so that we don’t lose our progress.
# ldm add-config alternate-domain-config
At this point, if we have done everything correctly, we can reboot the primary domain without disrupting service to the alternate domain. Doing pings during a reboot will show illustrate where we are in the build. Of course, you would have to have networking configured on the alternate domain, and don’t forget the simple stuff like mirroring your rpool and such, it would be a pity to go to all this trouble to not have a basic level of redundancy such as mirrored disks.
Test Redundancy
At this point, the alternate and the primary domain are completely independent. To validate this I recommend setting up a ping to both the primary and the alternate domain and rebooting the primary. If done correctly then you will not lose any pings to the alternate domain. Keep in mind that while the primary is down you will not be able to utilize the “control domain” in other words the only one which can configure and start/stop other domains.
