Boto3

 

🧠 Understanding Boto3: Overview

Boto3 is the official AWS SDK for Python, used to interact with AWS services like S3, EC2, Lambda, DynamoDB, etc.

---

⚙️ 1. boto3.client() vs boto3.resource() vs boto3.session()

✅ boto3.client(service_name, ...)

• Low-level client.

• Maps 1:1 to AWS service APIs.

• Returns response dicts (JSON-like).

• Example: client('s3')

✅ boto3.resource(service_name, ...)

• High-level abstraction.

• Uses Python objects.

• Easier for common operations (like bucket.upload_file(...))

⚠️ Not available for all AWS services.

✅ boto3.session.Session(...)

• Used to manage configuration: profiles, credentials, and regions.

• You can have multiple sessions, for example for multi-account or multi-region setups.

---

🔁 When to Use What

Feature	Use client()	Use resource()	Use session()
Needs raw API access	✅ Yes	❌ No	❌ Use session.client()
Object-based actions	❌ Too verbose	✅ Ideal	✅ For multi-profile access
Working across profiles	❌ Only default or env vars	❌ Same	✅ Fully supported
Need flexibility	✅ Advanced control	❌ Less control	✅ Multi-region and credential flexibility

---

✅ Example 1: Using boto3.client to List S3 Buckets

import boto3

s3_client = boto3.client('s3', region_name='us-east-1')

def list_buckets():
    response = s3_client.list_buckets()
    for bucket in response['Buckets']:
        print(f"- {bucket['Name']}")

Why use client?
We want direct access to AWS API to fetch raw data like bucket names.

---

✅ Example 2: Using boto3.resource to Upload File to S3

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket-name')

def upload_file():
    bucket.upload_file(Filename='file.txt', Key='uploaded_file.txt')

Why use resource?
This is a high-level operation (upload_file) which is easier with resource than calling put_object() manually with client.

---

✅ Example 3: Using boto3.session for Multiple Profiles

Let's say you have 2 AWS profiles: dev and prod.

import boto3

def get_instance_count(profile_name):
    session = boto3.Session(profile_name=profile_name)
    ec2 = session.client('ec2', region_name='us-east-1')
    
    instances = ec2.describe_instances()
    total = sum(len(reservation['Instances']) for reservation in instances['Reservations'])
    print(f"{profile_name} has {total} EC2 instance(s).")

get_instance_count('dev')
get_instance_count('prod')

Why use session?
Each session uses its own credentials and region. Useful for multi-account management.

---

✅ Example 4: Using boto3.session to Assume Role into Another Account

import boto3

def assume_role_and_list_s3(role_arn):
    base_session = boto3.Session()
    sts_client = base_session.client('sts')

    assumed_role = sts_client.assume_role(
        RoleArn=role_arn,
        RoleSessionName='CrossAccountSession'
    )

    credentials = assumed_role['Credentials']
    temp_session = boto3.Session(
        aws_access_key_id=credentials['AccessKeyId'],
        aws_secret_access_key=credentials['SecretAccessKey'],
        aws_session_token=credentials['SessionToken']
    )

    s3 = temp_session.client('s3')
    buckets = s3.list_buckets()
    for b in buckets['Buckets']:
        print(b['Name'])

assume_role_and_list_s3("arn:aws:iam::123456789012:role/SomeRole")

Why use session?
You can create temporary sessions with assumed roles — essential in enterprise, multi-account setups.

---

🧪 Quick Summary Table

Use Case	Method Used	Why?
List S3 buckets	boto3.client()	Raw API for precise data
Upload files to S3	boto3.resource()	High-level object methods
Switch between dev and prod accounts	boto3.session()	Supports multiple profiles
Cross-account access with STS assume role	boto3.session()	Use temporary credentials via STS

---

🧰 Pro Tip

Use session.client() or session.resource() like this:

session = boto3.Session(profile_name='dev')
s3_client = session.client('s3')

It gives you flexibility + cleaner multi-env support.

✅ 1. S3 File Operations (Upload, List, Download)

python
import boto3

# High-level resource
s3 = boto3.resource('s3')
bucket_name = 'my-demo-bucket'

# Upload a file
s3.Bucket(bucket_name).upload_file('local.txt', 'uploaded.txt')

# List objects
for obj in s3.Bucket(bucket_name).objects.all():
    print(f'File in bucket: {obj.key}')

# Download a file
s3.Bucket(bucket_name).download_file('uploaded.txt', 'downloaded.txt')

✅ Use resource for S3 when you want file operations, cleaner syntax, and auto-pagination.

---

✅ 2. EC2: Launch Instance, List Instances

python
import boto3

ec2 = boto3.resource('ec2')

# Launch new EC2 instance
instances = ec2.create_instances(
    ImageId='ami-0c55b159cbfafe1f0',
    InstanceType='t2.micro',
    MinCount=1,
    MaxCount=1
)

print("Launched instance ID:", instances[0].id)

# List running instances
for instance in ec2.instances.filter(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}]):
    print(instance.id, instance.instance_type, instance.state['Name'])

✅ boto3.resource('ec2') is great for managing instances in an object-oriented way.

---

✅ 3. EKS: List Clusters and Get Cluster Info

python
import boto3

eks = boto3.client('eks')

# List all EKS clusters
response = eks.list_clusters()
print("Clusters:", response['clusters'])

# Get details for a specific cluster
cluster_info = eks.describe_cluster(name='my-eks-cluster')
print("Cluster status:", cluster_info['cluster']['status'])

❗ EKS supports only client, not resource.

---

✅ 4. Lambda: List and Invoke a Function

python
import boto3
import json

lambda_client = boto3.client('lambda')

# List Lambda functions
functions = lambda_client.list_functions()
for func in functions['Functions']:
    print(func['FunctionName'])

# Invoke a function
response = lambda_client.invoke(
    FunctionName='my-function-name',
    InvocationType='RequestResponse',
    Payload=json.dumps({'key1': 'value1'}),
)

print("Function output:", response['Payload'].read().decode())

❗ client is required for AWS Lambda.

---

✅ 5. DynamoDB: Add Item, Query Table

python
import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('MyTable')

# Put item
table.put_item(Item={'id': '123', 'name': 'John Doe'})

# Get item
response = table.get_item(Key={'id': '123'})
print(response['Item'])

✅ resource is perfect for table access in DynamoDB.

EKS - 1

✅ Web App (3 Pods) ✅ MongoDB (3 Pods) ✅ AWS ALB Ingress ✅ Uses: Deployment, Service, ConfigMap, Secret, Ingress # =========================== # 1. Secret for MongoDB creds # =========================== apiVersion: v1 kind: Secret metadata: name: mongo-secret type: Opaque stringData: mongo-username: mongouser mongo-password: mongopass --- # =========================== # 2. ConfigMap for WebApp # =========================== apiVersion: v1 kind: ConfigMap metadata: name: webapp-config data: mongo-uri: mongodb://mongo-service:27017/mydb --- # =========================== # 3. MongoDB Deployment # =========================== apiVersion: apps/v1 kind: Deployment metadata: name: mongo spec: replicas: 3 selector: matchLabels: app: mongo template: metadata: labels: app: mongo spec: containers: - name: mongo image: mongo:6 ports: - containerPort: 27017 env: - name: MONGO_INITDB_ROOT_USERNAME valueFrom: secretKeyRef: name: mongo-secret key: mongo-username - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: secretKeyRef: name: mongo-secret key: mongo-password livenessProbe: httpGet: path: /healthz port: 80 initialDelaySeconds: 10 periodSeconds: 15 readinessProbe: httpGet: path: /readyz port: 80 initialDelaySeconds: 5 periodSeconds: 10 --- # =========================== # 4. MongoDB Service # =========================== apiVersion: v1 kind: Service metadata: name: mongo-service spec: selector: app: mongo ports: - protocol: TCP port: 27017 targetPort: 27017 type: ClusterIP --- # =========================== # 5. WebApp Deployment # =========================== apiVersion: apps/v1 kind: Deployment metadata: name: webapp spec: replicas: 3 selector: matchLabels: app: webapp template: metadata: labels: app: webapp spec: containers: - name: webapp image: your-ecr-repo/your-webapp:latest # Replace this ports: - containerPort: 80 env: - name: MONGO_URI valueFrom: configMapKeyRef: name: webapp-config key: mongo-uri - name: MONGO_USER valueFrom: secretKeyRef: name: mongo-secret key: mongo-username - name: MONGO_PASS valueFrom: secretKeyRef: name: mongo-secret key: mongo-password --- # =========================== # 6. WebApp Service # =========================== apiVersion: v1 kind: Service metadata: name: webapp-service labels: app: webapp spec: selector: app: webapp ports: - port: 80 targetPort: 80 type: ClusterIP --- # =========================== # 7. Ingress for WebApp via AWS ALB # =========================== apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: webapp-ingress annotations: kubernetes.io/ingress.class: alb alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]' spec: rules: - host: webapp.example.com # Update with your domain or local entry http: paths: - path: / pathType: Prefix backend: service: name: webapp-service port: number: 80 ✅ Deploy It kubectl apply -f k8s-all.yaml ✅ How External Access Works in EKS In EKS, to expose a service (like your web app) externally, you use: 🔸 Step 1: Internal Communication webapp-service (ClusterIP): Exposes port 80 inside the cluster. Pods can talk to it, but not accessible from outside. 🔸 Step 2: Expose via Ingress webapp-ingress defines routing rules for HTTP requests from outside. Uses the AWS Load Balancer Controller to provision an ALB (Application Load Balancer). ✅ Flow of External Access INTERNET │ ▼ [ALB] ← created by Ingress via AWS ALB Controller │ ▼ [Ingress] ──> matches path/host → routes to │ ▼ [webapp-service] (type: ClusterIP) │ ▼ [webapp pods] 🧪 Example: Customer Service Endpoints Let’s say your FastAPI app exposes: Endpoint HTTP Verb Description /customer GET List all customers /customer POST Add a new customer /customer/{id} GET Get specific customer /customer/{id} PUT Update a customer 🌐 Accessing Endpoints via ALB Assume your Ingress sets the host as: host: webapp.example.com And AWS ALB provides a DNS: ➡ a1b2c3d4e5f6.elb.us-east-1.amazonaws.com You can: 🔁 Option 1: Map DNS via /etc/hosts (Local Dev) sudo nano /etc/hosts Add: a1b2c3d4e5f6.elb.us-east-1.amazonaws.com webapp.example.com 🔁 Option 2: Use Route 53 DNS (Production) Point webapp.example.com to ALB DNS in a Route 53 Hosted Zone. 🧪 Test from Local Use curl or Postman: # List all customers curl http://webapp.example.com/customer # Get specific customer curl http://webapp.example.com/customer/123 # Add new customer (POST) curl -X POST http://webapp.example.com/customer \ -H "Content-Type: application/json" \ -d '{"name": "John", "email": "john@example.com"}' # Update customer curl -X PUT http://webapp.example.com/customer/123 \ -H "Content-Type: application/json" \ -d '{"email": "john.new@example.com"}' ✅ What Are Liveness and Readiness Probes in Kubernetes? Probes are used by Kubernetes to check the health of your application: Probe Type Purpose Liveness Probe Checks if the app is alive and should continue running Readiness Probe Checks if the app is ready to serve traffic ⚙️ What Happens Situation Liveness Status Readiness Status Effect App starts booting up ✅ alive ❌ not ready No traffic routed yet App fully ready ✅ alive ✅ ready Traffic routed App gets stuck (infinite loop) ❌ dead ❌ not ready Pod is restarted App healthy but DB is down ✅ alive ❌ not ready Pod not removed, but traffic not routed ✅ What is a Kubernetes Operator? A Kubernetes Operator is a method of automating the management of complex, stateful applications on Kubernetes using custom resources and custom controllers. examle - MondoDB, ArogoCD

Step Function

 

Type	Description
Task	Performs work by invoking a Lambda, ECS, or any supported AWS service
Choice	Adds conditional logic (like if/else or switch-case)
Wait	Waits for a specified time or timestamp before continuing
Pass	Passes input to output unchanged or with a fixed result
Succeed	Terminates the execution successfully
Fail	Terminates the execution with failure
Parallel	Executes multiple branches concurrently and waits for all to complete
Map	Applies a sub-workflow to each element of an array (similar to a loop)

{

  "Comment": "Simple Fund Transfer with Parallel Checks",

  "StartAt": "ParallelChecks",

  "States": {

    "ParallelChecks": {

      "Type": "Parallel",

      "Branches": [

        {

          "StartAt": "BalanceCheck",

          "States": {

            "BalanceCheck": {

              "Type": "Task",

              "Resource": "arn:aws:lambda:region:account-id:function:BalanceCheck",

              "Catch": [

                {

                  "ErrorEquals": ["States.ALL"],

                  "Next": "BalanceCheckFailed"

                }

              ],

              "End": true

            },

            "BalanceCheckFailed": {

              "Type": "Fail",

              "Cause": "Insufficient balance"

            }

          }

        },

        {

          "StartAt": "FraudCheck",

          "States": {

            "FraudCheck": {

              "Type": "Task",

              "Resource": "arn:aws:lambda:region:account-id:function:FraudCheck",

              "Catch": [

                {

                  "ErrorEquals": ["States.ALL"],

                  "Next": "FraudCheckFailed"

                }

              ],

              "End": true

            },

            "FraudCheckFailed": {

              "Type": "Fail",

              "Cause": "Fraud detected"

            }

          }

        }

      ],

      "Next": "TransferFunds"

    },

    "TransferFunds": {

      "Type": "Task",

      "Resource": "arn:aws:lambda:region:account-id:function:TransferFunds",

      "Next": "NotifySuccess",

      "Catch": [

        {

          "ErrorEquals": ["States.ALL"],

          "Next": "TransferFailed"

        }

      ]

    },

    "NotifySuccess": {

      "Type": "Task",

      "Resource": "arn:aws:lambda:region:account-id:function:NotifyCustomerSuccess",

      "End": true

    },

    "TransferFailed": {

      "Type": "Fail",

      "Cause": "Fund transfer failed"

    }

  }

}

Second

{

  "BalanceCheck": {

    "Type": "Task",

    "Resource": "arn:aws:lambda:region:account-id:function:BalanceCheck",

    "Catch": [

      {

        "ErrorEquals": ["States.ALL"],

        "Next": "BalanceCheckFailed"

      }

    ],

    "Next": "IsBalanceSufficient"

  },

  "IsBalanceSufficient": {

    "Type": "Choice",

    "Choices": [

      {

        "Variable": "$.balanceSufficient",

        "BooleanEquals": true,

        "Next": "WaitBeforeFraudCheck"

      }

    ],

    "Default": "BalanceCheckFailed"

  },

  "WaitBeforeFraudCheck": {

    "Type": "Wait",

    "Seconds": 30,

    "Next": "FraudCheck"

  },

  "FraudCheck": {

    "Type": "Task",

    "Resource": "arn:aws:lambda:region:account-id:function:FraudCheck",

    "Catch": [

      {

        "ErrorEquals": ["States.ALL"],

        "Next": "FraudCheckFailed"

      }

    ],

    "Next": "ApproveLoan"

  },

  "ApproveLoan": {

    "Type": "Task",

    "Resource": "arn:aws:lambda:region:account-id:function:ApproveLoan",

    "Next": "NotifyApproval"

  },

  "NotifyApproval": {

    "Type": "Task",

    "Resource": "arn:aws:lambda:region:account-id:function:NotifyApplicantApproval",

    "Next": "LogApproval"

  },

  "LogApproval": {

    "Type": "Task",

    "Resource": "arn:aws:lambda:region:account-id:function:LogApproval",

    "End": true

  },

  "BalanceCheckFailed": {

    "Type": "Fail",

    "Cause": "Insufficient balance or error during balance check"

  },

  "FraudCheckFailed": {

    "Type": "Fail",

    "Cause": "Fraud check failed"

  }

}