Setting Up Aqua Studio with Apache Flink SQL Gateway

This document outlines the steps to connect Aqua Studio to Apache Flink using the JDBC URL configured for the SQL Gateway.

Prerequisites

• Podman installed on your machine.
• Apache Flink image available from Docker Hub.

Step 1: Start Apache Flink with SQL Gateway

Start the JobManager Container

Run the Flink JobManager container and enable the SQL Gateway by passing environment variables:

podman run -d \
  --name flink-jobmanager \
  -p 8081:8081 \        # Port for the Flink Web UI
  -p 8082:8082 \        # Port for the SQL Gateway
  -e FLINK_SQL_GATEWAY_ENABLED=true \  # Enable the SQL Gateway
  -e FLINK_SQL_GATEWAY_PORT=8082 \     # Set the SQL Gateway port
  apache/flink:latest jobmanager

Start the TaskManager Container

Run the Flink TaskManager container, linking it to the JobManager:

podman run -d \
  --name flink-taskmanager \
  --link flink-jobmanager:jobmanager \  # Link to the JobManager
  -e JOBMANAGER_HOST=jobmanager \       # Specify the JobManager host
  -e JOBMANAGER_PORT=8081 \             # Specify the JobManager port
  apache/flink:latest taskmanager

Verify the SQL Gateway

1. Open your web browser.
2. Navigate to http://localhost:8082 to access the SQL Gateway web interface.

Step 2: Download the Flink JDBC Connector

1. Go to the Apache Flink download page.
2. Choose the desired version of Flink and download the binary package (e.g., flink-<version>-bin-scala_<scala_version>.tgz).

Extract the Flink Package

1. Once the download is complete, extract the package using:

   
   tar -xzf flink-<version>-bin-scala_<scala_version>.tgz
   

2. Change into the extracted directory:

   
   cd flink-<version>
   

Locate the JDBC Connector JAR

1. Inside the extracted Flink directory, navigate to the lib folder:

   
   cd lib
   

2. Look for the JDBC connector JAR file, typically named flink-sql-connector-jdbc-<version>.jar.

Step 3: Set Up Aqua Studio

Open Aqua Studio

1. Launch Aqua Studio on your computer.

Create a New Connection

1. In Aqua Studio, navigate to the Connections section.
2. Click on Add Connection or a similar option to create a new database connection.

Configure the Connection Settings

1. Connection Type: Select JDBC as the connection type.
2. JDBC URL: Enter the JDBC URL for the Flink SQL Gateway:

   
   jdbc:flink://localhost:8082
   

3. Driver Class: Specify the JDBC driver class:

   
   org.apache.flink.streaming.connectors.jdbc.JdbcConnection
   

4. Name: Give a meaningful name to your connection (e.g., "Flink SQL Gateway").

Configure the Driver

1. Add the JDBC connector JAR file (flink-sql-connector-jdbc-<version>.jar) to the connection configuration if prompted.

Test the Connection

1. Use the Test Connection feature in Aqua Studio to verify the connection.
2. Ensure you receive a confirmation message indicating success.

Step 4: Execute SQL Queries

Once connected, you can start writing and executing SQL queries against your Flink tables and data streams.

Example Query

You can execute a simple SQL query like:

SELECT * FROM your_table_name;

Troubleshooting

If you encounter any issues:

• Ensure the Flink SQL Gateway is running and accessible.
• Check Aqua Studio logs for any connection errors.
• Verify network settings and firewall rules that might block the connection.