ClickHouse

Configure a ClickHouse destination for the GoldRush Pipeline API to batch-insert structured blockchain data.

The ClickHouse destination writes structured pipeline output to a ClickHouse database using batch inserts. It supports configurable buffer sizes and flush intervals for throughput tuning.

Configuration

destination:
  type: "clickhouse"
  url: "clickhouse://host:8123/db"
  user: "${CH_USER}"
  password: "${CH_PASSWORD}"
  batch_size: 10000
  flush_interval_ms: 5000

Fields

Field	Type	Required	Default	Description
url	string	yes	—	JDBC connection URL
user	string	yes	—	Database user
password	string	yes	—	Database password
batch_size	int	no	10,000	Rows per batch insert
flush_interval_ms	long	no	5,000	Max time (ms) between flushes

How It Works

1. Rows are buffered in memory as they arrive from the pipeline.
2. When the buffer reaches batch_size or flush_interval_ms elapses since the last flush, a batch INSERT executes against the target table.
3. The INSERT SQL is dynamically generated from the field names present in each row.
4. All column names are automatically quoted in the generated SQL to handle reserved words.

Type Coercion

The destination maps pipeline data types to ClickHouse column types as follows:

Data Type	Database Handling
String	String column
Long	Int64 / UInt64
Integer	Int32 / UInt32
Double	Float64
Boolean	Bool / UInt8
byte[]	Bytes
null	Null