AWS Glue Python shell job timeout with custom Libraries

This is short post on Timeout errors faced using custom libraries with AWS Glue Python shell job.  I referred the steps listed in AWS docs to create a custom library , and submitted the job with timeout of 5 minutes.  But the job timed out without any errors in logs. Cloudwatch log reported following messages…

Identifying PostgreSQL Bloat and fixing it without downtime

MVCC (Multi-Version Concurrency Control) feature allows databases to provide concurrent access to data. This allows each SQL statement to see a snapshot of data as it was some time ago, regardless of the current state of the underlying data. This prevents statements from viewing inconsistent data produced by concurrent transactions performing updates on the same…

Saving AWS RDS and Redshift Billing Costs using Reserved Instances

In this post, we would understand what are reserved instances, when do you need them and various ways to monitor them in your environment. As more enterprises are embracing AWS cloud technologies, we need to understand AWS’s breadth of services and pricing options to right size our compute resources and run efficient infrastructure. Cost optimization…

Linux: ssh equivalence and SELinux

This is quick post summarizing issues encountered while setting up ssh equivalence on EC2 instance. I was setting up  RHEL7 EC2 instances and followed below procedure to setup ssh equivalence Generate rsa key-pair using ssh-keygen -t rsa on both hosts Copy the public keys to the remote server in authorized_keys file Modify file permission to…

Querying RDS PostgreSQL logs using Amazon Athena

PostgreSQL logs contains useful information which can help in troubleshooting performance issues,auditing and performing historical analysis. Some of items which can be logged are listed below Client connection and disconnection messages Failed DB connection request Database sql statement errors Database Fatal Errors Auditing Information Database Lock history Long running queries and Query plan logging High…