Step 1: Prepare Installation Files on a Machine with Internet Access
-
Download Trino Server
- Visit Trino Releases and download the latest Trino server
.tar.gz
file. - Example (for version 433):
wget https://repo1.maven.org/maven2/io/trino/trino-server/433/trino-server-433.tar.gz
- Visit Trino Releases and download the latest Trino server
-
Download Trino CLI
- Example:
wget https://repo1.maven.org/maven2/io/trino/trino-cli/433/trino-cli-433-executable.jar -O trino chmod +x trino
- Example:
-
Download Required Connectors
- Download the Kafka connector:
wget https://repo1.maven.org/maven2/io/trino/trino-kafka/433/trino-kafka-433.tar.gz
- Download the Protobuf decoder:
wget https://repo1.maven.org/maven2/io/trino/trino-kafka-protobuf-decoder/433/trino-kafka-protobuf-decoder-433.tar.gz
- Download the Kafka connector:
-
Transfer the Files to AWS EC2 Instance
- Use scp or SFTP to copy the downloaded files to the EC2 instance.
Step 2: Install and Configure Trino on AWS EC2
-
Extract Trino
tar -xvzf trino-server-433.tar.gz mv trino-server-433 /opt/trino
-
Create Trino Configuration Directories
mkdir -p /opt/trino/etc
-
Create
config.properties
coordinator=true node-scheduler.include-coordinator=true http-server.http.port=8080 discovery-server.enabled=true discovery.uri=http://localhost:8080
-
Create
node.properties
node.environment=production node.id=trino-node-1 node.data-dir=/var/trino/data
-
Create
jvm.config
-server -Xmx4G -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent
-
Configure Kafka Connector for Redpanda
mkdir -p /opt/trino/etc/catalog
- Create
/opt/trino/etc/catalog/kafka.properties
:connector.name=kafka kafka.nodes=<Redpanda_Broker>:9092 kafka.table-names-matching=.* kafka.default-schema=default kafka.messages-format=raw
- Create
-
Move Downloaded Connectors
tar -xvzf trino-kafka-433.tar.gz -C /opt/trino/plugin tar -xvzf trino-kafka-protobuf-decoder-433.tar.gz -C /opt/trino/plugin
Step 3: Set Up Local Schema Registry
-
Download and Transfer Schema Registry
wget https://packages.confluent.io/archive/7.5/confluent-community-7.5.0.tar.gz
- Transfer this file to the EC2 instance.
-
Install Schema Registry on EC2
tar -xvzf confluent-community-7.5.0.tar.gz mv confluent-7.5.0 /opt/schema-registry
-
Configure Schema Registry
- Create
/opt/schema-registry/etc/schema-registry/schema-registry.properties
:listeners=http://0.0.0.0:8081 kafkastore.bootstrap.servers=<Redpanda_Broker>:9092 kafkastore.topic=_schemas debug=false
- Create
-
Start Schema Registry
/opt/schema-registry/bin/schema-registry-start /opt/schema-registry/etc/schema-registry/schema-registry.properties &
-
Verify Schema Registry
curl http://localhost:8081/subjects
Step 4: Configure Trino to Use Local Schema Registry
-
Update
/opt/trino/etc/catalog/kafka.properties
connector.name=kafka kafka.nodes=<Redpanda_Broker>:9092 kafka.default-schema=default kafka.messages-format=protobuf kafka.schema-registry-url=http://localhost:8081
-
Restart Trino
/opt/trino/bin/launcher restart
-
Test Query on Redpanda
./trino --server http://localhost:8080 --catalog kafka --schema default SELECT * FROM kafka.your_protobuf_topic LIMIT 10;
Now, Trino should be connected to Redpanda, allowing SQL queries on your Protobuf topic.