Skip to content

Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud #2696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
odidev wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud #2696

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 17 additions & 12 deletions content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,25 @@
---
title: Deploy ClickHouse on Google Cloud C4A (Arm-based Axion VMs)

draft: true
cascade:
draft: true
title: Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud Axion (Arm-based C4A VMs)

minutes_to_complete: 30
minutes_to_complete: 50

who_is_this_for: This learning path is intended for software developers deploying and optimizing ClickHouse on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors.
who_is_this_for: This learning path is intended for software developers, data engineers, and platform engineers who want to build and benchmark a real-time analytics pipeline using ClickHouse on Linux/Arm64 environments, specifically Google Cloud C4A virtual machines powered by Axion processors.

learning_objectives:
- Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors)
- Install ClickHouse on a SUSE Arm64 (C4A) instance
- Verify ClickHouse functionality by starting the server, connecting via client, and performing baseline data insertion and simple query tests on the Arm64 VM
- Measure ClickHouse query performance (read, aggregation, and concurrent workloads) to evaluate throughput and latency on Arm64 (Aarch64)
- Provision an Arm-based SUSE SLES virtual machine on Google Cloud using C4A (Axion processors)
- Configure Google Cloud Pub/Sub for real-time log ingestion
- Deploy and validate ClickHouse on a SUSE Linux Arm64 (Axion) VM
- Build a streaming ETL pipeline using Apache Beam and Google Dataflow
- Ingest real-time Pub/Sub data into ClickHouse using Dataflow
- Validate end-to-end data flow from Pub/Sub to ClickHouse
- Perform baseline and analytical query benchmarking on ClickHouse running on Arm64
- Measure and report query latency (including p95) on Axion processors

prerequisites:
- A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled
- Basic familiarity with [ClickHouse](https://clickhouse.com/)
- Basic understanding of databases and SQL

author: Pareena Verma

##### Tags
Expand All @@ -30,7 +32,10 @@ armips:

tools_software_languages:
- ClickHouse
- clickhouse-benchmark
- Apache Beam
- Google Dataflow
- Google Cloud Pub/Sub
- Python 3.11

operatingsystems:
- Linux
Expand Down
2 changes: 1 addition & 1 deletion content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/baseline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: ClickHouse Baseline Testing on Google Axion C4A Arm Virtual Machine
weight: 5
weight: 7

### FIXED, DO NOT MODIFY
layout: learningpathall
Expand Down
Loading