AWS Wrangler (now officially AWS SDK for pandas) is an open-source Python library that simplifies working with AWS services like S3, Glue, Athena, Redshift, DynamoDB, and more. While it integrates seamlessly with pandas, it also supports other Python data libraries such as Apache Arrow, Modin, Ray, and PySpark, making it a versatile tool for AWS data processing.

AWS Wrangler for FinOps and Cost Optimization
FinOps and AWS cost optimization require working with vast amounts of data, including price lists, consumption data, inventory data, and organization data. Python is an excellent language for interacting with AWS data services, and AWS Wrangler makes this process even more efficient.

While there are edge cases where boto3 is still useful, in most cases, AWS Wrangler greatly simplifies and shortens your code—especially when incorporating Athena queries into Python workflows.

Key Features of AWS Wrangler

• Effortless Read & Write to S3 – Supports Parquet, CSV, JSON, and integrates with Glue Catalog.
• Optimized Queries on Athena – Pushes down filters to minimize scanned data and reduces Athena costs using query caching with CTAS (Create Table As Select).
• Leverage Glue Catalog – Streamlines schema management for better efficiency and transparency.
• Simplify DataFrame Operations – Read from and write to Parquet, CSV, or JSON files in S3 effortlessly.
• Works with Multiple AWS Data Services – Compatible with S3, Glue, Athena, Redshift, Timestream, OpenSearch, Neptune, QuickSight, CloudWatch Logs, DynamoDB, EMR, Secrets Manager, and RDS (Aurora, PostgreSQL, MySQL, SQL Server, Oracle).

The result? Optimized storage, faster queries, and lower costs.

Pro Tip for Maximum Cost Efficiency
Combine AWS Wrangler with tools like:

• Boto3
• Cost Explorer API
• S3 Inventory & S3 Lens Metrics exports
• Glue Data Catalog
• AWS Cost Optimization Hub

Using these together helps you analyze cost data and uncover optimization opportunities in real time. If you're already using pandas or other DataFrame libraries, AWS Wrangler feels like a natural extension—and if you’re managing AWS resources, it can make your workflow significantly more efficient.

Want to Optimize AWS Costs with AWS Wrangler?
Have you used AWS Wrangler in your cost optimization efforts? Whether you're just exploring its capabilities or actively using it, join the conversation over on LinkedIn, or connect with me directly on my LinkedIn page to discuss how it can streamline your workflows and reduce costs. Connect with us to explore AWS cost optimization strategies tailored to your needs!

In 2017, Netflix gave a talk at AWS re:Invent that left a lasting impression on me. The session, "Tooling Up for Efficiency: DIY Solutions @ Netflix", presented by Andrew Park and Sébastien Delarquier, shared how Netflix built in-house solutions to optimize cloud costs at scale.

This talk profoundly shaped my perspective on AWS cost management. Netflix laid out their approach to cloud management, with a particular focus on cost efficiency. They also made a compelling case for why large AWS users might benefit from developing their own cost optimization tools.

Trade-offs in Cloud Management
One key insight that I believe shapes any organization’s cost optimization efforts is understanding the trade-offs required when setting cloud management priorities:

• Innovation – Experimenting, iterating quickly, and developing new features.
• Reliability – Ensuring uptime, system resilience, and handling failure gracefully.
• Security – Protecting data, preventing breaches, and meeting compliance needs.
• Efficiency – Optimizing cloud costs and resource utilization.
  

No organization can maximize all four at once. If you’re going to prioritize something, something else must be deprioritized. Conversely, if everything is a priority, nothing truly is.
With this in mind, Netflix walked through how they manage cost efficiency and the tooling they developed to support it.

The Key Takeaway: Data-Driven Decision Making
Whether you’re using commercial tooling, cloud-native cost tools, DIY solutions, or a mix of all three, their approach still makes sense.
The biggest takeaway for me? Data-driven decision-making is essential for cloud cost optimization. The approach will feel familiar to many enterprise data teams—it’s a blend of business intelligence (BI), data analysis, data engineering, and data science.

A Personal Observation
Many of the capabilities Netflix outlined in their 2017 talk have since been incorporated into AWS’s native cost and billing tools, as well as the Well-Architected Framework. These tools are a great starting point—and for most shops, they get you most of the way there.

However, I’ve found that developing additional tooling for deeper data analysis, automation, and alerting can significantly enhance AWS’s built-in capabilities.

What’s Your Cloud Cost Management Strategy?
How does your organization approach AWS cost management? Do you prefer native AWS tools, commercial dashboards, DIY solutions, or a combination? Join the conversation over on LinkedIn, or connect with me directly on my LinkedIn page to share your experience or explore how we can help build a tailored cost optimization strategy for your organization.

After three years of public testing, Karpenter 1.0 was officially released in August, and since then, it has been rapidly evolving. Now at version 1.2.1, this open-source cluster autoscaler is ready for production workloads, providing a smarter way to manage Kubernetes infrastructure by dynamically selecting the most cost-efficient instance types without sacrificing performance.

​Karpenter is built for modern cloud-native environments, where flexibility is key. Whether you're scaling your infrastructure to handle peak traffic or optimizing resources during quieter periods, Karpenter helps you maximize efficiency and reduce costs.

Ready to Optimize Kubernetes at Scale? 
​Are you using Karpenter in your Kubernetes environment, or are you considering making the switch? Let’s explore how Karpenter can help optimize costs and enhance scalability for your AWS infrastructure. Join the conversation over on LinkedIn, or connect with me directly on my LinkedIn page to discuss best practices and implementation strategies for cost-effective Kubernetes management!