[Info-vax] VMWARE/ESXi Linux

[Dan Cross]

In article <674faad2$0$705$14726298 at news.sunsite.dk>,
Arne Vajhøj  <arne at vajhoej.dk> wrote:
>On 12/3/2024 7:41 PM, Dan Cross wrote:
>> In article <vio70q$e1fp$1 at dont-email.me>,
>> Arne Vajhøj  <arne at vajhoej.dk> wrote:
>>> ESXi has the vSphere suite of products. For many years the basic ESXi
>>> was actually free and customers only paid for the advanced vSphere
>>> stuff.
>>>
>>> For KVM there are many products to choose from. Redhat has
>>> Redhat OpenShift Virtualization (it used to be Redhat Virtualization,
>>> but it came under the OpenShift umbrella when containers took
>>> off). The big cloud vendors that may be managing millions of
>>> servers must have some custom tools for that. You gave a link
>>> to someone switching to the OpenNebula product. Proxmox VE is
>>> another option. Lots of different products with different
>>> feature sets to match different requirements.
>> 
>> It's unclear what you think that KVM is.  KVM requires a
>> userspace component to actually drive the VCPUs; that runs under
>> Linux, which is a "full host OS."  At least Google uses the same
>> management tools to drive those processes as it uses for the
>> rest of its production services (e.g., borg, etc).  The
>> userspace component for GCP is not QEMU, but rather, a Google
>> authored program.  However, it is in all-respects just another
>> google3 binary.
>
>That is the general model.
>
>central management server---(network)---management agent---hypervisor
>
>Details can vary but that is the only way to manage at scale.

If all you want to run on your host is VMs, maybe.

>And which is why the claim that the hypervisor has to come with
>a full host OS does not hold water for large production
>environments.

Define "full host OS."  My definition is a fully functinal,
general purpose operating system, with a full complement of
userspace tools, plus whatever applications the environment
it is running in require for management and maintenance.  This
includes the job scheduling daemon, but also system monitoring,
binary copies, upgrade agents, watchdogs, etc.  In this case,
we're talking about Linux.  In the Google environment, that's
the Google version of the kernel (prodkernel or increasingly
icebreaker: https://lwn.net/Articles/871195/), plus a set of
packages providing the usual complement of Unix-y command line
tools, borglet, the monitoring daemon, and a number of other
custom daemons.

>They just need the very basic OS, the virtualization service
>and the agent.

Not how they do it.

>Google could tailor down the Linux KVM they use to the very
>minimum if they wanted to. But I have no idea if they have
>actually bothered doing so.

They have not, nor would they.  There is substantial benefit at
Google scale to having a basic "node" architecture that's more
or less indistinguishable between the systems that run, say,
GMail and those that run GCP.  Plus all of Google's internal
services (globe-spanning distributed filesystems, databases,
etc).

	- Dan C.