AWS Day 49: Centralized Audit Logging with VPC Peering

Continuing my KodeKloud 100 Days of Cloud journey, Day 49 focused on building a simple centralized logging architecture using Amazon VPC, VPC Peering, EC2, IAM, S3, and Linux cron jobs.

The goal was to take a log file from an EC2 instance inside a private VPC, transfer it securely to an EC2 instance in a public VPC, and finally upload it to an S3 bucket.

The overall flow was:

Private EC2
    |
    | SCP
    v
Public EC2
    |
    | AWS CLI
    v
Private S3 Bucket

Step 1: Create the Public VPC

I created a new VPC named:

datacenter-pub-vpc

with the CIDR:

10.0.0.0/24

The existing private VPC was using:

10.10.0.0/16

Using different CIDR ranges is important because the two VPCs need non-overlapping address spaces for VPC Peering.

Step 2: Create the Public Subnet

Inside the new VPC, I created:

datacenter-pub-subnet

with:

10.0.0.0/25

This subnet was used for the public EC2 instance.

I also created the route table:

datacenter-pub-rt

and associated it with the public subnet.

Step 3: Configure Internet Gateway

To provide internet connectivity to the public subnet, I created:

datacenter-pub-ig

and attached it to:

datacenter-pub-vpc

Then I added the default internet route:

0.0.0.0/0
    |
    v
Internet Gateway

This allows the EC2 instance in the public subnet to communicate with the internet.

Step 4: Launch the Public EC2 Instance

I launched an Ubuntu EC2 instance named:

datacenter-pub-ec2

inside the public subnet.

The same SSH key pair used by the private instance was used so that I could establish connectivity between the two instances.

The basic connectivity path was:

datacenter-priv-ec2
        |
        | VPC Peering
        v
datacenter-pub-ec2

Step 5: Create the IAM Role for S3

Next, I created the IAM role:

datacenter-s3-role

The role was attached to the public EC2 instance.

The purpose of this role is to allow the EC2 instance to upload logs to S3 without storing AWS access keys directly on the server.

The required permission is essentially:

s3:PutObject

for the required S3 bucket.

This is a better approach than putting an AWS access key and secret key inside a script or cron job.

Step 6: Create the S3 Bucket

I created the private S3 bucket:

datacenter-s3-logs-18106

The bucket is used as the final centralized destination for the logs.

The required object path is:

datacenter-priv-vpc/boot/boots.log

So the final structure looks like:

datacenter-s3-logs-18106
└── datacenter-priv-vpc
    └── boot
        └── boots.log

Step 7: Configure VPC Peering

To allow private and public VPC resources to communicate, I created:

datacenter-vpc-peering

between:

datacenter-priv-vpc

and:

datacenter-pub-vpc

The important part is that VPC Peering itself does not automatically add routes.

Both route tables need routes for the opposite VPC CIDR.

For the private route table:

Destination: 10.0.0.0/24
Target: VPC Peering Connection

For the public route table:

Destination: 10.10.0.0/16
Target: VPC Peering Connection

This allows traffic between the two VPCs through the peering connection.

Step 8: Test SSH Connectivity

After configuring the peering connection and routes, I tested connectivity from the private instance to the public instance.

I used the existing SSH key:

ssh -i datacenter-key.pem ubuntu@10.10.1.12

The first connection prompted for host verification, after which the host was added to known_hosts.

The important point here was that communication between the instances was happening through private IP addresses and VPC Peering, rather than sending the traffic over the public internet.

Step 9: Configure Cron on the Private Instance

The private instance contains the log file:

/var/log/boots.log

I configured a cron job to periodically copy this file to the public EC2 instance using scp.

Example:

* * * * * scp -i /root/datacenter-key.pem /var/log/boots.log ubuntu@10.0.0.4:/tmp/boots.log

The important part is the flow:

Private EC2
    |
    | SCP over private IP
    v
Public EC2

This avoids exposing the private instance directly to the internet.

Step 10: Configure Cron on the Public Instance

Once the log reaches the public EC2 instance, another cron job uploads it to S3.

For example:

* * * * * aws s3 cp /tmp/boots.log s3://datacenter-s3-logs-18106/datacenter-priv-vpc/boot/boots.log

Because the EC2 instance has the appropriate IAM role, the AWS CLI can use the instance role credentials automatically.

No static AWS credentials are required in the cron job.

Conclusion

AWS Day 49 was a practical exercise in connecting AWS networking, security, storage, and Linux automation.

The most useful part was seeing the complete flow from a log generated on a private EC2 instance to centralized storage in S3:

Private EC2 → VPC Peering → Public EC2 → S3

It was another hands-on step in understanding how AWS networking and DevOps automation can be combined to build a reliable logging workflow.