Flink 1.20.2 on EC2

Step 1: Download Flink 1.20.2 on Your Local Machine

1. Download Apache Flink 1.20.2 from the official site:
   🔗 Flink 1.20.2 Downloads - https://dlcdn.apache.org/flink/flink-1.20.1/flink-1.20.1-src.tgz
2. Choose the binary release (e.g., flink-1.20.2-bin-scala_2.12.tgz).
3. Download Flink SQL Gateway from the same page if it's not included in the binary package.

---

Step 2: Transfer Flink to EC2

Use scp to transfer files to your EC2 instance:

bash
scp -i your-key.pem flink-1.20.2-bin-scala_2.12.tgz ec2-user@your-ec2-ip:/home/ec2-user/

Alternatively, use WinSCP or an internal network if available.

---

Step 3: Extract and Configure Flink

1. SSH into your EC2 instance:

   bash
   ssh -i your-key.pem ec2-user@your-ec2-ip
   

2. Extract Flink:

   bash
   tar -xvzf flink-1.20.2-bin-scala_2.12.tgz
   cd flink-1.20.2
   

3. Modify conf/flink-conf.yaml:

   bash
   nano conf/flink-conf.yaml
   

   Update the following configurations:

   yaml
   jobmanager.rpc.address: localhost
   taskmanager.numberOfTaskSlots: 4  # Adjust based on EC2 resources
   parallelism.default: 2
   

4. Set environment variables:

   bash
   echo "export FLINK_HOME=/home/ec2-user/flink-1.20.2" >> ~/.bashrc
   echo "export PATH=$FLINK_HOME/bin:$PATH" >> ~/.bashrc
   source ~/.bashrc
   

---

Step 4: Start Flink Services

1. Start the JobManager:

   bash
   ./bin/start-cluster.sh
   

---

Step 5: Configure and Start Flink SQL Gateway

1. Edit conf/sql-gateway-defaults.yaml:

   bash
   nano conf/sql-gateway-defaults.yaml
   

   Add these configurations:

   yaml
   server:
     port: 8083
   execution:
     target: "local"
   

2. Start Flink SQL Gateway:

   bash
   ./bin/sql-gateway.sh start
   

---

Step 6: Verify Installation

• Check if SQL Gateway is running:

  bash
  netstat -tulnp | grep 8083
  

• Start SQL Client:

  bash
  ./bin/sql-client.sh
  

• Run a test query:

  sql
  SHOW TABLES;