[Info-vax] VMWARE/ESXi Linux

[Dan Cross]

In article <vinbg2$3sjr$7 at dont-email.me>,
Arne Vajhøj  <arne at vajhoej.dk> wrote:
>On 12/3/2024 10:55 AM, Dan Cross wrote:
>> In article <vin939$3sjr$5 at dont-email.me>,
>> Arne Vajhøj  <arne at vajhoej.dk> wrote:
>>> On 12/3/2024 10:36 AM, Dan Cross wrote:
>>>> In article <vin597$3sjr$2 at dont-email.me>,
>>>> Arne Vajhøj  <arne at vajhoej.dk> wrote:
>>>>> On 12/2/2024 11:57 PM, Lawrence D'Oliveiro wrote:
>>>>>> On Tue, 3 Dec 2024 03:09:15 -0000 (UTC), Waldek Hebisch wrote:
>>>>>>>    From what you wrote seem that ESXi is more similar to Xen than to
>>>>>>> KVM+qemu, that is ESXi and Xen discourage running unvirtualized programs
>>>>>>> while in KVM+qemu some (frequently most) programs is running
>>>>>>> unvirtualized and only rest is virtualized.
>>>>>>
>>>>>> I think that dates back to the old distinction between “type 1” and “type
>>>>>> 2“ hypervisors. It’s an obsolete distinction nowadays.
>>>>>
>>>>> No.
>>>>>
>>>>> If you look at what is available and what it is used for then you will
>>>>> see that what is labeled type 1 is used for production and what is
>>>>> labeled type 2 is used for development. It matters.
>>>>
>>>> No, that has nothing to do with it.
>>>
>>> Yes. It has.
>>>
>>> The question was whether the type 1 vs type 2 distinction is obsolete.
>> 
>> As I've posted on numerous occasions, at length, citing primary
>> sources, the distinction is not exact; that doesn't mean that it
>> is obsolete or useless.
>
>The post I was replying to called it obsolete. So that was the topic
>of my post.

Yes.  I somewhat agree; I just think your argument is predicated
on falsehoods.  I don't disagree with your conclusion, I
disagree with your framing.

>>> The fact that "what is labeled type 1 is used for production and what is
>>> labeled type 2 is used for development" proves that people think it
>>> matters.
>> 
>> That seems to be something you invented: I can find no serious
>> reference that suggests that what you wrote is true,
>
>Is is your experience that people do their development on ESXi/KVM
>and run their production on VMWare Player/VirtualBox?

Some people do, yes.  Many others run production workloads on
Bhyve and KVM.

>:-)
>
>People do development on VMWare Player/VirtualBox and run
>production on ESXi/KVM.

Some people do.  Some people do development on z/VM and deploy
on z/VM.  Some do development on bare metal and deploy on KVM or
Bhyve (or z/VM).

Some people do development on VMs hosted on ESXi.

>> hard to see how it "proves" anything.  KVM is used extensively
>> in production and is a type-2 hypervisor, for example.
>
>When I wrote "is labeled" I am talking about what the
>authors and the industry in general are calling it.

I see no evidence for, and plenty of contradicting inf

>In that sense KVM is a labeled a type 1 hypervisor. I can
>find Redhat links if you don't believe me.

I know.  I already said that it was claimed that it was a type-1
HV.  Here, I'll save you the trouble of finding the RedHat link:
https://www.redhat.com/en/topics/virtualization/what-is-a-hypervisor

Here's the relevant section:

|A type 1 hypervisor, also referred to as a native or bare metal
|hypervisor, runs directly on the host's hardware to manage
|guest operating systems. It takes the place of a host operating
|system and VM resources are scheduled directly to the hardware
|by the hypervisor. 

Yes, that's Goldberg's definition.

|This type of hypervisor is most common in an enterprise data
|center or other server-based environments.

Ok, sure; that's marketing speak, but whatever.

|KVM, Microsoft Hyper-V, and VMware vSphere are examples of a
|type 1 hypervisor. KVM was merged into the Linux kernel in
|2007, so if you're using a modern version of Linux, you already
|have access to KVM. 

Here's the problem.  How does KVM match the definition of a
type-1 hypervisor listed above?  In particular, we know that it
delegates the functionality for resource management and
scheduling to Linux.  Indeed, actually causing a VCPU to run is
done by executing a system call from a userspace process using
e.g. QEMU or CrosVM or Firecracker or some other userspace HV
component.

It then goes on to say:

|A type 2 hypervisor is also known as a hosted hypervisor, and
|is run on a conventional operating system as a software layer
|or application.

Yup.  That's exactly what KVM does.

So yes.  RedHat calls KVM a type-1 hypervisor, but that doesn't
make it so.  The industry writ large commonly accepts it as a
type-2 HV.

>That you consider it to be type 2 does not really matter.

Not just me.  Do a literature search and tell me what the
consensus is about whether KVM is a type-1 or type-2 hypervisor.

Here's an example from the book "Harda and Software Support for
Virtualization", by Edouard Bugnion, Jason Nieh and Dan Tsafrir
(Morgan and Claypool, 2017).  From page 7:

|We note that the emphasisis on resource allocation, and not
|whether the hypervisor runs in privileged or non-privileged
|mode.  In particular, a hypervisor can be a type-2 even when it
|runs in kernel-mode, e.g., Linux/KVM and VMware Workstation
|operate this way.  In fact, Goldberg assumed that the
|hypervisor would always be executing with supervisor
|privileges.

In fact, we can go deeper.  If we go back to the 2007 KVM paper
by Kitty et al from the Ottawa Linux Symposium
(https://www.kernel.org/doc/ols/2007/ols2007v1-pages-225-230.pdf)
we can see this text in the abstract:

|The Kernel-based Virtual Machine, or kvm, is a new Linux
|subsystem which leverages these virtualization extensions to
|add a virtual machine monitor (or hypervisor) capability to
|Linux. Using kvm, one can create and run multiple virtual
|machines. These virtual machines appear as normal Linux
|processes and integrate seamlessly with the rest of the system.

This is precisely what type-2 hypervisors do.  Note also this,
from section 3 of that paper:

|Under kvm, virtual machines are created by opening a device
|node (/dev/kvm.) A guest has its ownmemory, separate from the
|userspace process that created it. A virtual cpu is not
|scheduled on its own, however.

So we see that guests are created by opening a device file, and
furthermore, that VCPU scheduling is not done by KVM (an
important criteria for a type-1 hypervisor is that it handles
VCPU scheduling).  And while a guest does own its own memory,
inspection of the KVM implementation shows that this is done by
using memory primitives provided by Linux.

So despite what some intro text on a RedHat web page says, KVM
does not meet any of the criteria for being a type-1 HV, while
it does meet the criteria for being a type-2 HV.

>> used extensively in production, and claims to be a type-2
>> hypervisor (even though it more closely resembles a type-1 HV).
>
>True.
>
>The type 1 for production and type 2 for development does
>not hold in the mainframe world.

It doesn't really hold anywhere; that's the point.

	- Dan C.