[Info-vax] VMWARE/ESXi Linux

[Dan Cross]

In article <slrnvkgb2b.2dr8a.mwilson at daenerys.home.mattwilson.org>,
Matthew R. Wilson <mwilson at mattwilson.org> wrote:
>On 2024-11-28, Lawrence D'Oliveiro <ldo at nz.invalid> wrote:
>> On Wed, 27 Nov 2024 22:24 +0000 (GMT Standard Time), John Dallman wrote:
>>
>>> In article <vi84pm$6ct6$4 at dont-email.me>, ldo at nz.invalid (Lawrence
>>> D'Oliveiro) wrote:
>>>>
>>>> On Wed, 27 Nov 2024 16:33:56 -0500, David Turner wrote:
>>>>
>>>>> I keep being told that VMWARE is not an OS in itself.
>>>>> But it is... based on Ubuntu Kernel....  stripped down but still
>>>>> Linux
>>>> 
>>>> And not even using the native KVM virtualization architecture that is
>>>> built into Linux.
>>> 
>>> History: VMware ESXi was released in 2001 and KVM was merged into the
>>> Linux kernel in 2007.
>>
>> In other words, VMware has long been obsoleted by better solutions.
>
>Please explain how ESXi is obsolete, and how KVM is a better solution.

I wouldn't bother trying to argue with him: he's a known troll.

>Both KVM and ESXi use the processor's VT-d (or AMD's equivalent, AMD-Vi)
>extensions on x86 to efficiently handle instructions that require
>hypervisor intervention. I'm not sure how you'd judge which one is a
>better solution in that regard. So the only thing that matters, really,
>is the virtualization of everything other than the processor itself.

So Goldberg defined two "types" of hypervisor in his
dissertation: Types 1 and 2.  Of course, this is an over
simplification, and those of us who work on OSes and hypervisors
understand that these distinctions are blurry and more on a
continuum than hard and fast buckets, but to a first order
approximation these categories are useful.

Roughly, a Type-1 hypervisor is one that runs on the bare metal
and only supports guests; usually some special guest is
designated as a trusted "root VM".  Xen, ESXi, and Hyper-V are
examples of Type-1 hypervisors.

Again, roughly, a Type-2 hypervisor is one that runs in the
context of an existing operating system, using its services and
implementation for some of its functionality; examples include
KVM (they _say_ it's type 1, but that's really not true) and
PA1050.  Usually with a Type-2 HV you've got a userspace program
running under the host operating system that provides control
functionality, device models, and so on.  QEMU is an example of
such a thing (sometimes, confusingly, this is called the
hypervisor while the kernel-resident component, is called the
Virtual Machine Monitor, or VMM), but other examples exist:
CrosVM, for instance.

>KVM is largely dependent on qemu to provide the rest of the actual
>virtual system.

I think that QEMU is what one _often_ uses, but it doesn't have
to be.  I mentioned CrosVM above, which works with KVM, but
other examplex exist: Google, Amazon, and AliBaba all use KVM on
their cloud offerings, but at least neither Google nor Amazon
use QEMU; I don't know about AliBaba but I suspect they have
their own.  (Microsoft of course uses Hyper-V.)

>qemu's a great project and I run a ton of desktop VMs
>with qemu+KVM, but it just doesn't have the level of maturity or
>edge-case support that ESXi does. Pretty much any x86 operating system,
>historical or current, _just works_ in ESXi.  With qemu+KVM, you're
>going to have good success with the "big name" OSes...Windows, Linux,
>the major BSDs, etc., but you're going to be fighting with quirks and
>problems if you're trying, say, old OS/2 releases. That's not relevant
>for most people looking for virtualization solutions, and the problems
>aren't always insurmountable, but you're claiming that KVM is a "better"
>solution, whereas in my experience, in reality, ESXi is the better
>technology.
>
>(As an aside, VMWare's _desktop_ [not server] virtualization product,
>VMWare Workstation, looks like it's making moves to use KVM under the
>hood, but they have said they will continue using their own proprietary
>virtual devices and drivers, which is really what sets VMWare apart from
>qemu. This is a move they've already made on both the Windows and Mac OS
>version of VMWare Workstation if I understand correctly [utilizing
>Hyper-V and Apple's Virtualization framework]. This makes sense... as I
>said, the underlying virtualization of the processor is being handled by
>the VT-x capabilities of the processor whether you're using VMWare,
>VirtualBox, KVM, etc., so when running a desktop product under Linux,
>you may as well use KVM but you still need other software to build the
>rest of the virtual system and its virtual devices, so that's where
>VMWare and qemu will still differentiate themselves. None of this is
>relevant for ESXi, though, because as has been pointed out earlier in
>the thread, it is not running on Linux at all, so VMKernel is providing
>its own implementation of, essentially, what KVM provides in the Linux
>kernel.)

Well, what KVM provides+a whole lot more.  ESXi is effectively
its own operating system, even though it's marketed as a type-1
HV.

>qemu and KVM have the huge advantage that they are open source and free
>software, of course, whereas ESXi (and vCenter) are closed source and
>expensive (barring the old no-cost ESXi license).
>
>But ESXi just works. It's solid, it has a huge infrastructure around it
>for vSAN stuff, virtual networking management, vMotion "just works," I
>find the management interface nicer than, say, Proxmox (although Proxmox
>is an impressive product), etc.
>
>It's sad to see Broadcom is going to do everything they can to drive
>away the VMWare customer base. VMWare will lose its market-leader
>position, FAR fewer people will learn about it and experiment with it
>since Broadcom killed the no-cost ESXi licenses, and popularity of
>Proxmox is going to skyrocket, I suspect. Which isn't a bad thing --
>when open source solutions get attention and traction, they continue to
>improve, and as I said earlier, Proxmox is already an impressive product
>so I look forward to its future.
>
>But make no mistake: VMWare was -- and I'd say still is -- the gold
>standard for virtualization, both on the server (ESXi) and the
>workstation (VMWare Workstation). VMWare's downfall at the hands of
>Broadcom will 100% be due to Broadcom's business practices, not
>technology.

Yup, it's a bit sad, though it does open up a lot of market
opportunities for other players.

>I'm a bit of a free software zealot, yet even I still use ESXi for my
>"real" servers. I do look forward to eventually replacing my ESXi boxes
>with Proxmox for philosophical reasons, but I'm in no rush.

Check out Bhyve; it's very nice.

	- Dan C.