[Info-vax] Another "Rendez-vous autour de VMS" in France (report)

[Arne Vajhøj]

On 12/4/2022 9:24 PM, Richard Maher wrote:
> On 4/12/2022 11:38 am, Arne Vajhøj wrote:
>> Small and medium size solutions use ActiveMQ/ArtemisMQ or RabbitMQ 
>> (probably mostly depending on whether they prefer product names that 
>> start with J or not).
>>
>> Large size solutions go for Kafka. Kafka is really the only option
>> for extreme workloads.
>>
>> A few of the big Kafka users are:
>>
>> LinkedIn - 100 clusters with 4000 nodes processing 7 trillion messages 
>> per day (2019)
>>
>> Pinterest - 50 clusters with 3000 nodes and a peak processing of 40
>> million messages per second (2021).
> 
> SO they use Message Queues to mirror/distribute messages/copies?

ActiveMQ/ArtemisMQ or RabbitMQ are typical just for async transactional
processing.

The big Kafka users use it for distributing data in general and
the highest volume is probably from the transactional system to
the analytical system.

If you go to one of those web sites then they log everything you
do and analyze your behavior to show you the right ads. But that
stuff does not move by magic - it flows through Kafka.

So when you move every page shown, every graphics shown, every click
made by hundreds of millions of users then it becomes humongous
amount of data.

Arne