CloudPro

Critical Kubernetes Vulnerability: Ingress-nginx CVE-2025-1974CloudPro #85: Kubernetes v1.33 sneak peekMulti-cloud compliance in a multi-jurisdictional worldThe cloud has become more like a fog, obscuring lurking compliance risks.Read full article🔐 Cloud SecurityIngress-nginx CVE-2025-1974: Critical Kubernetes VulnerabilityRecently patched vulnerabilities in ingress-nginx (used by over 40% of Kubernetes clusters) could allow attackers to extract Secrets or take over your entire cluster — even without admin access. Update immediately to avoid exposure.How Red Canary Detects Cloud Threats at Scale: A 6-Phase PipelineRed Canary shares its cloud detection pipeline built to sift through billions of telemetry events. Six phases: streamline enrichment, correlation, and surfacing of real threats, useful for anyone building or evaluating cloud threat detection systems.4 Patterns for Fine-Grained Access Control in Kubernetes with Amazon Verified PermissionsThis article shows how to use AVP for Kubernetes RBAC across 4 real-world patterns—multi-tenant clusters, namespace-level control, team-based access, and dynamic policy enforcement.Critical 0-Days in Fluent Bit: Are Your Logs a Threat Vector?Two high-impact vulnerabilities in Fluent Bit (a widely used log forwarder) allow memory corruption and DoS. If you use Fluent Bit in production, especially exposed endpoints, patch ASAP.Compliance as Code with CheckovThis article walks through building a custom compliance policy for AWS security groups using Python and Checkov. It shows how to codify tagging rules, test them using HCL and unit tests, and integrate them into CI/CD pipelines—ideal for teams enforcing org-specific IaC standards.[Sponsored] Join cybersecurity thought leader David Linthicum for a special fireside chat to learn how to use AI and ML to unify your data strategies, uncover hidden cloud costs, and overcome the limitations of your traditional data protection in public cloud environments.⚙️ Infrastructure & DevOpsGrafana 11.6 Released: Dashboards, Cron-based Annotations, Better SecurityGrafana 11.6 adds one-click data links in visualizations, Cron-based annotations, improved geomap performance using WebGL, and experimental LBAC for metrics data.Master Multi-State Terraform Projects with AtmosAtmos is a powerful Terraform wrapper built by CloudPosse to manage complex, multi-state deployments with ease. It walks through how Atmos organizes components and stacks using YAML, automates state handling, and integrates workflows to bring up entire environments with just a couple of commands.How to refactor code with GitHub CopilotThis article shows how GitHub Copilot can help you clean up and refactor your code more easily—by suggesting improvements, creating reusable modules, and simplifying large, messy functions. With smart prompts and planning, Copilot can do a lot of the heavy lifting for you.How to Use Terraform Import Block for Importing ResourcesThis article explains how Terraform’s import block (introduced in v1.5) lets you declaratively import existing resources, like S3 buckets, EC2 instances, and Azure resource groups, directly into your Terraform config. No more separate CLI commands or manual state juggling.Use Testkube + Keptn to block bad deploys in K8s PipelinesIntegrate Testkube with Keptn to enforce automated testing before each deployment stage. You’ll learn how to set up a quality gate that halts deployments if tests fail—using pre-deployment tasks and Testkube workflows to validate your app in Kubernetes. It’s a practical way to catch issues early and keep broken code out of production.📦 Kubernetes & Cloud NativeKubernetes v1.33 sneak peekKubernetes v1.33 introduces support for user namespaces, in-place resource resizing for Pods, and major API deprecations. If you're managing clusters, this is a must-read before the April release.[Sponsored] Google Workspace isn't built to stop modern threats—Material is. See the difference.How to Manage Existing Helm Charts with Terraform (Without Breaking Everything)This article explains how to integrate existing Helm charts—like Metrics Server—into Terraform without causing conflicts or duplicate deployments. It walks through setting up the Helm provider, importing the chart, and handling common issues (like resource drift) that show up when migrating from other tools like ArgoCD.Live Migrate KubeVirt VMs Without Dropping a PacketKubeVirt just got live migration support via container-native virtualization. You can now migrate running VMs across Kubernetes nodes without network disruptions or packet drops. A huge win for stateful workloads in K8s.The Hidden Gaps in Kubernetes Audit Logs and How They Can Break Your DetectionsThere are real-world problems with relying solely on Kubernetes audit logs for security, like missing events, inconsistent log formats across providers (like GKE vs. EKS), and limited control over audit policies, all of which can lead to missed attacks and broken detections. It also offers practical strategies to fill these gaps with additional logging and monitoring sources.Why a Giant K8s Cluster (with vCluster) Might Be Your Best BetConsolidating everything into a single large Kubernetes cluster boosts efficiency, reduces overhead, and simplifies operations. It also tackles the downsides like blast radius and multi-tenancy by introducing vCluster, a tool that creates fully isolated virtual clusters within a host cluster. The result? You get the best of both worlds: centralized control with team-level autonomy.🔍 Observability & SRENew Cloud Trace features to troubleshoot latency and errors | Google Cloud BlogGoogle Cloud’s new Trace Explorer makes debugging services easier with span heatmaps, percentile duration charts, and filters — all powered by BigQuery. Essential for SREs handling production latency issues.Grafana Loki 3.4: Unified Storage, Smarter Sizing, and the Promtail-to-Alloy ShiftThis article covers the major updates in Grafana Loki 3.4—from adopting Thanos as the standard storage client to new cluster sizing guidance based on real-world usage. It also highlights better support for out-of-order log ingestion and the official merging of Promtail into Grafana Alloy, giving teams a unified telemetry collector with OTLP support.Rethinking SLOs: Slice by Team, Defend by Design, Align on OutcomesThis article explores how to make service-level objectives (SLOs) more effective by splitting them across teams and designing for failure. Instead of alerting everyone for every issue, teams can define what they own, set their own performance budgets, and use strategies like caching or retries to absorb downstream failures. The result? Less noise, clearer accountability, and a better user experience.A Practical Guide to Using OpenTelemetry and the OTel Collector for Full-Stack ObservabilityThis article explains how to use OpenTelemetry and the OTel Collector to collect logs, metrics, and traces from your apps and infrastructure. It shows how to configure receivers for Redis, MySQL, and NGINX, and how to export data to backends like Prometheus or Jaeger. The goal is to help you build a flexible, scalable observability pipeline using open standards.How a Concurrency Bug Caused 3200% CPU UtilizationThe author debugged a Java program using 3,200% CPU and traced it to multiple threads writing to an unguarded TreeMap, causing data corruption and an infinite loop inside the red-black tree structure. The bug wasn’t immediately visible because exceptions were swallowed silently by thread pools. Through experiments, they confirmed how concurrent modification can break TreeMap, not by crashing it, but by corrupting its internal structure into cycles.🌐 Industry, Tools, AI & OtherGemini Code Assist: A Framework for AI Dev Tools AdoptionGoogle Cloud proposes a four-phase model (Adoption → Trust → Acceleration → Impact) to roll out AI code tools like Gemini. It offers clear metrics to measure ROI from day one.A step-by-step guide to writing a System Design documentSQL Noiris a game where you solve crimes with SQL queries and uncover evidence through data.Stelviois a Python library that simplifies cloud infrastructure management and deployment.OpenSSF announces initial release of the open source project security baselineCheers,Shreyans SinghEditor-in-ChiefM365 Protection: Guided Lab Experience. See how Rubrik's M365 backup functionality saves timeCheck it out NowForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Amazon Inspector Expands to Lightweight Containers and MoreCloudPro #84: WAF Just Got Smarter: Now It Sees the “#” in Your URLsMulti-cloud compliance in a multi-jurisdictional worldThe cloud has become more like a fog, obscuring lurking compliance risks.Read full article🔐 Cloud SecurityHow Red Canary Detects Cloud Threats at ScaleRed Canary processes over 6 billion telemetry records a day to find threats in cloud control planes like unauthorized access, API abuse, and data exfiltration. They break detection into six key steps: Ingest, Standardize, Combine, Detect, Suppress, Respond.WAF Just Got Smarter: Now It Sees the “#” in Your URLsAWS WAF can now match against URI fragments (the part after # in a URL), allowing more granular security rules to block unauthorized access or detect bots.Scanning S3 for Malware? Now Available in GovCloud TooGuardDuty Malware Protection is now live in AWS GovCloud regions, letting teams scan S3 uploads for threats and automatically isolate suspicious files.IAM Access Analyzer Now Speaks IPv6AWS IAM Access Analyzer now supports IPv6 through new dual-stack endpoints—helping teams monitor and secure resource access in IPv6-enabled environments.Amazon Inspector Expands to Lightweight Containers and MoreAmazon Inspector now supports scanning scratch, distroless, and Chainguard containers, plus detects vulnerabilities in widely used ecosystems like Go, JDK, WordPress, and more.It also flags discontinued OSes to help prioritize security fixes.Sponsored: DevSecOps is dead… or is it? Discover why your security strategy might be failing—and what to do about it.⚙️ Infrastructure & DevOpsFrom Serverless to CDK: One Dev’s Full Migration PlaybookDavid Behroozi shares his complete journey migrating a live API from the Serverless Framework to AWS CDK, including how to safely import existing resources like DynamoDB and CloudWatch LogGroups.9+ Terraform Tools That Make Your Code Cleaner, Safer, and Production-ReadyEssential tools for managing Terraform code: from linters like TFLint and documentation generators like terraform-docs to security scanners like Checkov and cost estimators like Infracost.A Terraform Toolbox for Real-World IaC TeamsThis post breaks down the best tools to supercharge your Terraform pipeline—static analysis, automated docs, pre-commit hooks, and more.Terraform Just Made Importing Resources Easier: Here's HowWith Terraform 1.5+, the new import block lets you declaratively bring existing resources (like S3, EC2, or Azure RGs) into your config, no CLI hackery required.10 Terraform Config Structures That Scale With Your Team and InfraRyan Cartwright breaks down 10 Terraform setup patterns—from single env to multi-tenant SaaS, microservices, and multi-cloud. Clear examples, pros/cons, and use cases make this a go-to resource for scaling Terraform cleanly.Sponsored: M365 Protection: Guided Lab Experience. See how Rubrik's M365 backup functionality saves time📦 Kubernetes & Cloud NativeZero-Downtime Kubernetes deployments on AWS with EKSGlasskube’s engineers dissect the nuances of AWS Load Balancer Controller behavior and explain why rolling updates often trigger 502/504 errors. Their fix? A trio of battle-tested solutions: inject Pod Readiness Gates to sync with ALB health checks, implement graceful shutdown in Go, and bake in termination delays to handle load balancer lag.Amazon EKS auto mode with TerraformMarcin Cuber walks through provisioning a fully-managed EKS Auto Mode cluster using Terraform—no node groups, minimal networking hassle, and serverless-like scaling. Includes full Terraform config for VPC, IAM, and EKS, plus a clean demo deploying a 2048 game app with ALB via Ingress.Kubernetes CPU limits: Best practices for Kubernetes CPU managementDevtron’s Rupin Solanki lays out why CPU limits can sabotage performance: unnecessary throttling, wasted resources, and misleading stability. TL;DR: stop setting CPU limits unless you absolutely need them. Use requests instead.Provisioning Kubernetes on Bare Metal using AWS EKS-AnywhereYou’ll learn how PXE booting, DHCP/TFTP, and YAML-driven workflows come together to spin up nodes with Bottlerocket OS. Includes multi-yaml config samples, hardware CSV explanations, and notes on local testing with VirtualBox.My Kubernetes pods keep crashing with “CrashLoopBackOff” but I can’t find any log10-step guide to diagnosing CrashLoopBackOff issues—when logs are missing and clues are scarce. Covers probes, exit codes, resource limits, and debugging techniques like kubectl exec with sleep overrides. Also includes lesser-known tricks like using ephemeral debug containers.🔍 Observability, Monitoring & SREOpenTelemetry collector deployment modes in KubernetesThe ultimate guide to OpenTelemetry visualizationObserving Lambdas using the OpenTelemetry Collector Extension Layer | OpenTelemetryGrafana Loki 3.4: Standardized storage config, sizing guidance, and Promtail merging into AlloyScaling Prometheus from single node to enterprise-grade observability🌐 Industry, Tools, AI & OtherHow GitLab lost 300GB of production dataSQLite or PostgreSQL? It isn't very easy!Admineris a full-featured database management tool written in PHP. It consists of a single file ready to deploy to the target server.xlskubectlis a spreadsheet to control your Kubernetes cluster.Beginner’s guide to software architecture with design patternsCheers,Shreyans SinghEditor-in-ChiefM365 Protection: Guided Lab Experience. See how Rubrik's M365 backup functionality saves timeCheck it out NowForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Infamous DevOps roadmapCloudPro #83: Kubernetes Spotlight on SIG AppsAddressing AI-generated misinformationHow to minimize the risks and consequences of flawed inference from AI models.Read full article⭐MasterclassInfamous DevOps roadmapKubernetes Open Source Limits & Requests Configuration OptimizationA guide to modern Kubernetes network policiesUsing Python Virtual Environments in DockerHow to terminate Go programs elegantly – a guide to graceful shutdowns🔍Secret KnowledgeHow Meta Enforces Purpose Limitation at ScaleWhy I Use Nim Instead of Python for Data ProcessingConvert OpenTelemetry Traces to Metrics using SpanMetrics ConnectorWhat happens when bucket.grantRead() in AWS CDKPreventing the Risk of Request Collapsing in Web Caching⚡TechwaveKubernetes Spotlight on SIG AppsAWS Pi Day 2025: Data foundation for analytics and AISecuring Datadog’s Cloud Infrastructure: Our Playbook and MethodologyScale Unstructured Text Analytics with Batch LLM InferenceAmazon EKS now envelope encrypts all Kubernetes API data by default🛠️HackhubKardinal: lightest-weight way to spin up dev and test environments in KubernetesKubeblocks: control plane software that runs and manages databases, message queues on K8s.Flipt: Enterprise-ready, GitOps enabled, CloudNative feature management solutionKubecolor: Colorize your kubectl outputAWS-mine: AWS honey token managerCheers,Shreyans SinghEditor-in-ChiefYour Salesforce Data, Your Responsibility: Best Practices for Data ProtectionLearn MoreForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Troy Hunt: Processing 23 Billion Rows of ALIEN TXTBASE Stealer LogsCloudPro #82: Kubernetes History Inspector (KHI) visualizes cluster logs: by Google CloudYour Salesforce Data, Your Responsibility: Best Practices for Data ProtectionLearn More⭐MasterclassStateful apps in Kubernetes. From history and fundamentals to operatorsStateful applications in Kubernetes require additional attention due to their need to persist data across instances and reboots, unlike stateless applications.Backstage on KubernetesIn this article, you'll learn how to integrate Backstage with Kubernetes, first by running Backstage outside the cluster using the Kubernetes API, and then deploying it directly on the cluster with the official Helm chart.Manage secrets in AWS EKS with AWS Secrets Manager securelyAWS Secrets Manager integrates with EKS through the AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI Driver.Your guide to observability engineering in 2024In 2024, an observability engineer's role is multifaceted, requiring expertise in data pipelines, system analysis, and troubleshooting to maintain and optimize complex, distributed systems.The complete guide to serverless appsWhile the term implies the absence of servers, it actually refers to a model where developers don't need to manage server infrastructure. Instead, the cloud provider handles server management🔍Secret KnowledgeHow Stripe’s Document Databases Supported 99.999% Uptime with Zero-Downtime Data MigrationsThis reliability is largely due to their custom-built database infrastructure, called DocDB, which is an extension of MongoDB Community.Anomaly Alerting in PrometheusUsing Prometheus with Istio, we can set up a generic anomaly detection system for response times that applies to all services running on a mesh.The ROI of improving and investing in DORADORA Metrics provide a comprehensive view of your software team's performance, helping identify areas for improvement and measure the return on investment (ROI) of these enhancements.AWS Managed KMS Keys and their Key Policies: Security Implications and Coverage for AWS ServicesAWS Managed KMS Keys are encryption keys managed by AWS but used within your own AWS account, often applied as default keys for various services.Attack Paths Into VMs in the CloudThis post reviews how attackers might exploit VMs and offers strategies for organizations to secure their environments.⚡TechwaveKubernetes History Inspector (KHI) visualizes cluster logs: by Google CloudTroy Hunt: Processing 23 Billion Rows of ALIEN TXTBASE Stealer LogsGet insights from multimodal content with Amazon Bedrock Data Automation, now generally availableOptimizing incident management with AIOps using the Triangle SystemNew Cloud Trace features to troubleshoot latency and errors🛠️HackhubSQLSync: collaborative offline-first wrapper around SQLite. It is designed to synchronize web application state between users, devices, and the edge.lambda_helpers_metrics:A library that simplifies sending custom metrics to CloudWatch using EMF (Embedded Metric Format).cloudysetup: Automate cloud environment setup using generative AI and AWS Cloud Control API.kftray: A cross-platform system tray application for managing multiple kubectl port-forward commands, with support for UDP and proxy connections through k8s clusterskeycloak: Open Source Identity and Access Management For Modern Applications and ServicesCheers,Shreyans SinghEditor-in-ChiefProtect Data Privacy and Optimize AI Models with Tonic TextualLLMs have tapped all of publicly available data. The last mile training of models requires private data. Use private data without compromising security. Redact, label, and prep freetext for LLM ingestion or data pipelines.Start Free TrialForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Announcing CDK Garbage CollectionCloudPro #82: NFTables mode for kube-proxy | KubernetesThe Self-Taught Cloud Computing EngineerBreaking into cloud computing can feel overwhelming, especially when juggling AWS, Azure, and GCP. The Self-Taught Cloud Computing Engineer stands out because it provides a structured, hands-on approach to mastering all three major cloud platforms.Whether you're aiming for certifications, career growth, or just a deeper understanding, this book walks you through real-world projects and practical skills that matter. If you're serious about leveling up in cloud computing, this is one to check out.GET IT⭐MasterclassFinding Vulnerabilities at Scale: How a JPEG processing vulnerability led to discovering security flaws in major projects like Chromium and WINE.Simplifying Multi-Environment Kubernetes Deployments: Kluctl automates deployments, integrates with Helm and Kustomize, and reduces manual intervention.How to Structure a Terraform Project: The tutorial covers approaches like monorepo and polyrepo setups for managing environments and modules.What is Inference Parallelism and how it worksGuide to deploy SpinKube with WASM on Taikun CloudWorksGo faster! Optimizing Golang for performance and scaleFlatcar brings Container Linux to the CNCF Incubator347 Million reasons to manage HashiCorp Vault as codeSpotlight on Kubernetes upstream training in JapanOpenSSF Expands secure development course with Interactive Labs🔍Secret KnowledgeDeploy Azure Resources from GitLab with No Secrets Using OpenTofu: Learn how to deploy Azure resources using GitLab pipelines with OpenID Connect, all without managing secrets.Promoting Terraform Changes from DEV to PROD: Learn how to use workspaces for separate states and environment folders to clearly separate multi-envs deployments.Implementing Compliant Secrets with AWS Secrets Manager:This tutorial walks you through building detailed access policies and introduces a Terraform module to automate and simplify policy management.Terraform Stacks with Azure:This guide covers dynamic credentials, creating stacks with multiple components, and using orchestration rules for automatic deployment approvals.Why Falco’s new response engine is a game changer for open source cloud native securityThere and back again: Port forwarding with mirrordKCD UK slides: Brownfield realities, platforms orchestration & app devsHow to enter Kubestronaut orbit and beyondA beginner’s guide to progressive delivery of a cloud native applicationFlatcar accepted into CNCF at incubating level⚡TechwaveNFTables mode for kube-proxy | KubernetesAnnouncing CDK Garbage CollectionAnnouncing the general availability of AWS .NET OpenTelemetry librariesAnnouncing new models, customization tools, and enterprise agent upgrades in Azure AI FoundryEmpowering innovation: The next generation of the Phi familyNew Terraform provider for Oracle Database@Google CloudNew Cloud Trace features to troubleshoot latency and errorsAutomating IT Network support with watsonx and Juniper’s Mist AIAmazon ECS increases the CPU limit for ECS tasks to 192 vCPUsAWS Network Firewall introduces automated domain lists and insights🛠️Hackhub:Booster Framework: create event-driven backend microservices that focus on extreme development productivityyunionio/cloudpods: A cloud-native open-source unified multi-cloud and hybrid-cloud platformvmware-tanzu/velero: Backup and migrate Kubernetes applications and their persistent volumestsypuk/aws-client-monitor: advanced monitoring of aws client (both aws cli and any language aws sdk)sjramblings/ebsight: EBSight: Intelligent EBS Volume Analyzerturbot/tailpipe: select * from logs; Tailpipe is an open source SIEM for instant log insights, powered by DuckDB. Analyze millions of events in seconds, right from your terminal.aws-samples/aws-health-events-insight: centralized approach to store and analyze AWS Health events(PHD,SHD)dannysteenman/vscode-iam-service-principal-snippets: VS Code extension that provides autocompletion of all AWS services that can be used as Service Principals in your IAM policies.aws-samples/bedrock-engineer: Autonomous software development agent apps using Amazon Bedrock, capable of customize to create/edit files, execute commands, search the web, use knowledge base, use multi-agents, generative images and more.awslabs/StsSamlDriverCheers,Shreyans SinghEditor-in-ChiefForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Applying SRE principles to your MLOps pipelinesCloudPro #81: Securing DeepSeek and other AI systems with Microsoft Security⭐MasterclassAmazon S3 now supports appending data to an objectThe dangers of Terraform automation platformsHyrum's law in GolangCloud dev environmentsA Deep Look into Our New Massive Multitenant ArchitectureTesting with Go and PostgreSQL: Learn how to use ephemeral PostgreSQL instances in Go to simplify tests, reduce resource usage, and improve CI efficiency. Practical tips included.Creating an AMI with Image Builder:Learn to use Packer and Terraform to create pipelines, manage custom AMIs, and streamline automation.Designing a Zero Downtime Migration:Complete guide on Docker: Learn to build and deploy your distributed applications easily to the cloud with Docker.SQL Style Guide: This SQL Style Guide outlines best practices for writing clear and maintainable SQL code.🔍Secret KnowledgeSemantic Versioning to Simplify Release Management:Learn how to use Semantic Versioning with semantic-release and GitHub Actions in your AWS CDK project.How to Use Blocks in Ansible Playbooks:How to use Ansible blocks to group tasks, handle errors gracefully, and manage cleanup operations effectively in your playbooks.Monitoring PM2 in production: Learn how to monitor PM2-managed Node.js apps in New Relic using Flex, capturing key metrics like CPU, memory, and logs with a streamlined setup.Infra as Code with CDK for Terraform: Learn how Zip’s security team used Python CDK for Terraform to enforce security guardrails on AWS infrastructure.Ingesting CloudWatch Logs into OpenSearch: Sample code to showcase ingestion of Amazon CloudWatch logs into Amazon OpenSearch Serverless.AWS Cost for Home Assistant: This tutorial showcases how to get the month-to-date cost and forecast cost and make it available on Home Assistant.Installing Windows XP in DOSBox-X: Learn how to install Windows XP on DOSBox-X, even though it’s not officially supported.Solve Missing AWS Resources in Terraform: The awscc provider, using AWS's Cloud Control API, helps overcome resource gaps.Optimizing PostgreSQL Queries at Scale: Tips to troubleshoot inefficient queries and resource bottlenecks.Getting Started with Terraform Stack: Simplify deployments across environments like dev, staging, and production without complex CI/CD setups.⚡TechwaveIntroducing the AWS Trust CenterAWS CloudTrail network activity events for VPC endpoints now generally availableAWS Verified Access support for non-HTTP resources is now generally availableSecuring DeepSeek and other AI systems with Microsoft SecurityApplying SRE principles to your MLOps pipelinesGrafana Cloud updates: Exemptions in Adaptive Logs, GPU monitoring in AI Observability, and moreIntroducing RHEL AI 1.4: Powering the Next Wave of Generative AI InnovationAlibaba, Datadog, and Quesma Join Forces on Go Compile-Time InstrumentationEnrich Your On-Call Experience With Observability Data at Your Fingertips by Using Datadog On-CallCheers,Shreyans SinghEditor-in-ChiefAWS Cloud ProjectsLately, I've been on the lookout for books that don’t just explain AWS concepts but actually help you build something useful. AWS Cloud Projects caught my attention because it takes a hands-on approach—walking through projects that range from hosting a website to building AI-powered applications. If you're like me and prefer learning by doing, check this out.GET ITForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Policy as code in Kubernetes: security with seccomp and network policiesCloudPro #80: Policy as code in Kubernetes: security with seccomp and network policiesWe want to makeCloudProeven better for professionals like you! Take ourquick 10-minute surveyand help shape the content we create. As a thank you, you'll receive a freePackteBook (worth $20) and have the option to participate in a paid user interview.Take the Survey NowThe Ultimate Linux Shell Scripting Guide: Automate, Optimize, and Empower tasks with Linux Shell ScriptingI’ve always believed that knowing how to write solid shell scripts is an underrated superpower. Whether you're automating tasks, troubleshooting servers, or just making your daily workflow smoother, scripting saves time and headaches. The Ultimate Linux Shell Scripting Guide caught my eye because it doesn’t just focus on Bash—it also dives into Zsh and even PowerShell on Linux. If you’re looking to sharpen your command-line skills and build real-world automation, this one is worth a read.GET IT⭐MasterclassPolicy as code in Kubernetes: security with seccomp and network policiesThe Kubernetes introduction I wish I had when I started: A complete guideA practitioner’s guide to wide eventsLonghorn backup and restoreKubernetes CI/CD pipelines with CircleCI and DevtronMoving off Heroku, slowlyDo you really need Redis?Discovering hidden vulnerabilities in Portainer with CodeQLWhat’s new with Robinhood, our in-house load-balancing serviceWhat is Helm in Kubernetes? A complete guide🔍Secret KnowledgeReducing the cost of a Google Cloud Dataflow Pipeline by over 60%Reflections on IaC using TerraformHow to Improve Your DevOps AutomationToo Many MicroservicesEnhance Argo CD observability: A step-by-step guide to integrating Prometheus metrics into the UIParsing all the data with open source tools: Unstructured and PgaiMonitoring MySQL using Prometheus Exporter and GrafanaThat time when KinD stopped working in GitHub CodespacesIngesting F1 Telemetry UDP real-time data in AWS EKS⚡TechwaveAWS CDK is splitting Construct Library and CLINew Amazon S3 Tables: Storage optimized for analytics workloadsSpanner Graph is now GAAnnouncing Gen AI Toolbox for Databases by Google CloudGrafana 11.5 release: easily share Grafana dashboards and panels, secure frontend code for plugins, and moreDigma Adds Ability to Predict Coding Issues to Observability PlatformThe Cloud Controller Manager Chicken and Egg ProblemAmazon EC2 now supports automated recovery of Microsoft SQL Server with VSSGKE’s faster cluster upgrades under the hoodAWS CloudTrail network activity events for VPC endpoints now generally availableCheers,Shreyans SinghEditor-in-ChiefAWS Cloud ProjectsLately, I've been on the lookout for books that don’t just explain AWS concepts but actually help you build something useful. AWS Cloud Projects caught my attention because it takes a hands-on approach—walking through projects that range from hosting a website to building AI-powered applications. If you're like me and prefer learning by doing, check this out.GET ITForward to a Friend📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Automating Windows Server Management with PowerShellWe want to make CloudPro even better for professionals like you! Take our quick 10-minute survey and help shape the content we create. As a thank you, you'll receive a free Packt eBook (worth $20) and have the option to participate in a paid user interview.Take the Survey NowCloudPro: Special IssueHey there,As cloud professionals, we are always looking for ways to improve our skills and build solutions that are scalable, secure, and efficient. While regular news and updates keep us informed, sometimes it's good to take a deep dive into topics that matter.That’s why we’re bringing you this special issue of CloudPro, where we explore two carefully selected books that provide practical, hands-on learning experiences.The first book, Platform Engineering for Architects, takes a deep dive into building and maintaining internal developer platforms. We’ll explore a hands-on section on Building a Self-Service Internal Developer Platform with Terraform, helping you understand Infrastructure as Code (IaC) in action.The second book, Windows Server 2025 Administration Fundamentals, goes beyond basic administration to teach PowerShell automation techniques for managing Windows Server environments. We've included a technical excerpt on Automating Active Directory User Management, a crucial skill for IT administrators.The third book, Cloud Observability with Azure Monitor, provides in-depth guidance on monitoring cloud infrastructure. We’ve included a highly practical excerpt on Configuring Azure Monitor for Real-Time Log Analysis, showing how to use KQL queries to detect performance issues before they impact users.If you’re serious about learning by doing, this issue is for you. Dive in and explore!Platform Engineering for ArchitectsBuilding a Self-Service Internal Developer Platform (IDP)One of the primary objectives of platform engineering is to provide an internal developer platform (IDP) that automates infrastructure provisioning. Below is a Terraform snippet to provision a Kubernetes cluster using Infrastructure as Code (IaC):provider "aws" { region = "us-east-1"}resource "aws_eks_cluster" "example" { name = "example-cluster" role_arn = aws_iam_role.example.arn vpc_config { subnet_ids = [aws_subnet.example1.id, aws_subnet.example2.id] }}After applying this configuration, developers can interact with the cluster using kubectl with minimal setup.Why This Matters:Automating platform setup reduces cognitive load on developers, improves consistency, and ensures security best practices are followed.GET IT HEREWindows Server 2025 Administration FundamentalsAutomating Windows Server Management with PowerShellPowerShell scripting is crucial for automating administrative tasks in Windows Server. Below is a script to create, configure, and secure a new Active Directory user automatically:# Define user parameters$UserName = "jdoe"$Password = ConvertTo-SecureString "SecureP@ssw0rd" -AsPlainText -Force# Create the user in Active DirectoryNew-ADUser -Name "John Doe" -SamAccountName $UserName -UserPrincipalName "$UserName@example.com" -Path "OU=Users,DC=example,DC=com" -AccountPassword $Password -Enabled $true# Add user to a security groupAdd-ADGroupMember -Identity "Developers" -Members $UserNameWhy This Matters:Manually managing user accounts is inefficient and error-prone. Automating it ensures compliance and operational efficiency.GET IT HERECloud Observability with Azure MonitorConfiguring Azure Monitor for Real-Time Log AnalysisAzure Monitor helps track and analyze cloud infrastructure performance. The following Kusto Query Language (KQL) query identifies virtual machines experiencing high CPU usage:Perf| where ObjectName == "Processor" and CounterName == "% Processor Time"| summarize AvgCPU = avg(CounterValue) by Computer, bin(TimeGenerated, 5m)| where AvgCPU > 80| order by AvgCPU descTo run this query in Azure Log Analytics, follow these steps:=> Navigate to Azure Monitor > Logs.=> Select your Log Analytics Workspace.=> Paste the KQL query and click Run.Why This Matters: Proactive monitoring ensures infrastructure stability. Automating performance alerts reduces downtime and enhances reliability.GET IT HERE📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Designing Scalable Microservices with KubernetesCloudPro: Special IssueHey there,As cloud professionals, we are always looking for ways to improve our skills and build solutions that are scalable, secure, and efficient. While regular news and updates keep us informed, sometimes it's good to take a deep dive into topics that matter.That’s why we’re bringing you this special issue of CloudPro, where we explore two carefully selected books that provide practical, hands-on learning experiences.The first book, AWS Cloud Projects, takes a step-by-step approach to building real-world cloud solutions. We’ll walk through a key project—Deploying a Serverless Application on AWS Lambda with Terraform—to help you understand infrastructure as code (IaC) in action.The second book, Solutions Architect’s Handbook, goes beyond the basics to teach cloud-native architecture best practices. We’ve included an in-depth section on Designing Scalable Microservices with Kubernetes to help you optimize your deployments for performance and efficiency.If you’re serious about learning by doing, this issue is for you. Dive in and explore!AWS Cloud ProjectsLately, I've been thinking a lot about the value of hands-on learning. There's something about actually building projects that sticks with you far longer than just reading concepts. That’s why when I came across AWS Cloud Projects, I knew it was worth sharing with you.This book doesn’t just explain AWS concepts—it walks you through real-world implementations, step by step. Whether you’re spinning up cloud infrastructure, deploying AI-powered applications, or optimizing security, the projects in this book serve as practical blueprints.One particular chapter stood out: Deploying a Serverless Application on AWS Lambda with Terraform. Here’s a detailed excerpt to give you a strong foundation:"In this project, we’ll set up a serverless API using AWS Lambda and API Gateway, all provisioned through Terraform. Infrastructure as Code (IaC) allows us to automate deployments, ensuring repeatability and reducing manual effort.Step 1: Define the Lambda FunctionWe start by defining our Lambda function using Terraform. Below is a basic Terraform configuration to deploy a function:resource "aws_lambda_function" "my_lambda" { function_name = "serverless_api" handler = "index.handler" runtime = "nodejs14.x" role = aws_iam_role.lambda_exec.arn filename = "lambda.zip"}Step 2: Configure API GatewayAPI Gateway allows our Lambda function to be exposed as an HTTP endpoint:resource "aws_api_gateway_rest_api" "api" { name = "serverless_api" description = "API Gateway for our Lambda function"}Step 3: Deploying the InfrastructureTo apply these changes, we use:terraform initterraform apply -auto-approveBy following these steps, you’ll have a fully operational serverless API deployed on AWS using Terraform.If you're someone who learns best by building, AWS Cloud Projects is a must-read. It’s the kind of book that makes learning AWS both practical and engaging.GET IT HERESolutions Architect's HandbookCloud-Native Architecture: Scaling Beyond LimitsAnother book I recently found valuable is Solutions Architect’s Handbook. It goes beyond the basics of cloud architecture and explores scalability, security, and generative AI in real-world applications. If you’re serious about designing scalable cloud systems, this book is a gem.Here’s a deep dive into Designing Scalable Microservices with Kubernetes:"When designing microservices at scale, Kubernetes provides a resilient, self-healing platform. But to truly optimize performance, we must consider three key factors: resource allocation, observability, and network efficiency.Resource Optimization with Horizontal Pod Autoscaler (HPA)Using HPA ensures workloads dynamically adjust to demand:apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: my-app-hpaspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu targetAverageUtilization: 50Observability with Prometheus & GrafanaMonitoring plays a crucial role in scaling applications. The book explains how to integrate Prometheus and Grafana for real-time insights into resource consumption and request rates.apiVersion: monitoring.coreos.com/v1kind: ServiceMonitormetadata: name: my-app-monitorspec: selector: matchLabels: app: my-app endpoints: - port: metricsNetwork Optimization using IstioIstio allows fine-grained traffic control, helping balance workloads efficiently.apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata: name: my-appspec: hosts: - "my-app.example.com" http: - route: - destination: host: my-app subset: v1With these techniques in place, you can scale workloads efficiently without unnecessary costs. Solutions Architect’s Handbook provides a deeper look into how enterprises design cloud-native applications for high availability and scalability.GET IT HERE📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Coldplay + Microsoft AI CloudPro #79: Google Cloud, AWS, and Azure have collaborated to introduce Kube Resource Orchestrator (kro) AWS Cloud Projects Lately, I've been on the lookout for books that don’t just explain AWS concepts but actually help you build something useful. AWS Cloud Projects caught my attention because it takes a hands-on approach—walking through projects that range from hosting a website to building AI-powered applications. If you're like me and prefer learning by doing, check this out. GET IT ⭐Masterclass The Kubernetes gap in CNAPP Unlock Kubernetes Savings with Kubecost’s Automated Actions How WebAssembly components extend the frontiers of Kubernetes to multi-cloud, edge, and beyond How to migrate an observability platform to open-source and cut costs 🔍Secret Knowledge Implementing GitOps with Kubernetes: Automate, manage, scale, and secure infrastructure and cloud-native applications on AWS and Azure Complete Guide to Logging in Golang with slog Scaling Prometheus with Thanos Automated container CVE and vulnerability patching using Trivy and Copacetic Self-signed Root CA in Kubernetes with k3s, cert-manager and traefik ⚡Techwave IGoogle Cloud, AWS, and Azure have collaborated to introduce Kube Resource Orchestrator (kro) Coldplay + Microsoft AI Amazon S3 Tables now support 10,000 tables per table bucket Insights into Azure's DDoS defense DeepSeek-R1 models now available on AWS 🛠️Hackhub Production-ready Kubernetes distribution for both public and private cloud Application Performance Monitoring System Graceful shutdown and Kubernetes readiness / liveness checks for any Node.js HTTP applications Toolkit for Integrating with your kubernetes dev environment more efficiently Backup your Kubernetes Stateful Applications Cheers, Shreyans Singh Editor-in-Chief The Ultimate Linux Shell Scripting Guide: Automate, Optimize, and Empower tasks with Linux Shell Scripting I’ve always believed that knowing how to write solid shell scripts is an underrated superpower. Whether you're automating tasks, troubleshooting servers, or just making your daily workflow smoother, scripting saves time and headaches. The Ultimate Linux Shell Scripting Guide caught my eye because it doesn’t just focus on Bash—it also dives into Zsh and even PowerShell on Linux. If you’re looking to sharpen your command-line skills and build real-world automation, this one is worth a read. GET IT Forward to a Friend ⭐MasterClass: Tutorials & Guides The Kubernetes gap in CNAPP Initially, CNAPPs focused on integrating various cloud security tools and supporting enterprises during early cloud adoption. As a result, their Kubernetes protection often lacks depth and focuses mainly on surface-level issues like container vulnerabilities, without addressing the complexities of Kubernetes clusters, such as control plane security or runtime policies. This has led to a false sense of security in cloud environments, as CNAPPs fail to offer robust Kubernetes-specific features. Unlock Kubernetes Savings with Kubecost’s Automated Actions Kubecost's new automated actions help users save money in their Kubernetes environments by optimizing resource usage with minimal effort. With features like automated request sizing, cluster turndown, and namespace turndown, Kubecost identifies inefficiencies like over-provisioned containers and shuts down unused clusters or namespaces. Users can set schedules for automating these actions, reducing waste and freeing up resources. How WebAssembly components extend the frontiers of Kubernetes to multi-cloud, edge, and beyond WebAssembly (Wasm) components enable Kubernetes to extend seamlessly across multi-cloud, edge, and other distributed environments by providing a lightweight, portable way to run applications across any architecture. Wasm components, similar to containers, can be written in various languages and connected through shared APIs, allowing for greater flexibility and efficiency. By integrating with Kubernetes through wasmCloud, a Wasm-native orchestrator, organizations can enhance their cloud-native setups without changing existing infrastructure. How to migrate an observability platform to open-source and cut costs Migrating an observability platform to open-source can significantly reduce costs while maintaining control over telemetry data, but it requires careful planning and execution. This process involves identifying essential telemetry data, selecting an open-source stack for logs, metrics, and traces, conducting proofs-of-concept (POCs) across different systems, and ensuring compatibility with various architectures, such as microservices. The migration also includes reconfiguring alerts and dashboards, validating the new setup, and updating related systems like notification and incident management tools. 🔍Secret Knowledge: Learning Resources Implementing GitOps with Kubernetes: Automate, manage, scale, and secure infrastructure and cloud-native applications on AWS and Azure This book provides practical guidance on using GitOps to automate and manage Kubernetes deployments in cloud-native environments like AWS and Azure. It explains core GitOps principles, tools like Argo CD and Flux, and strategies for implementing CI/CD pipelines. The book also covers infrastructure automation with Terraform, security best practices, and observability while addressing cultural transformations in IT for GitOps adoption. By the end, readers will have skills to apply GitOps in scaling, monitoring, and securing Kubernetes deployments efficiently. Complete Guide to Logging in Golang with slog In Golang, structured logging can be efficiently implemented using the `slog` package, introduced in version 1.21. `slog` allows for more organized and detailed log entries by formatting logs as key-value pairs, making them easier to search, filter, and analyze. The package provides flexibility with logging levels (like Debug, Info, Warn, and Error) and supports both text-based and JSON-formatted output. Key components include Loggers, Records, and Handlers, which define how logs are created, stored, and processed. Scaling Prometheus with Thanos Scaling Prometheus with Thanos allows for long-term storage, cost savings, and a global view of metrics in large environments. While Prometheus is great for short-term monitoring, it struggles with long-term storage and querying across multiple clusters. Thanos extends Prometheus by using components like Thanos Query, Sidecar, and Store Gateway to enable scalable, highly available storage through object stores, reducing Prometheus's resource consumption. It also supports downsampling to optimize storage and query performance. Automated container CVE and vulnerability patching using Trivy and Copacetic Automating container vulnerability patching with Trivy and Copacetic (copa) helps protect your applications from potential attacks by scanning and patching container images automatically. Trivy scans container images for vulnerabilities, generating a report in JSON format, while Copacetic reads this report and patches the container image based on detected vulnerabilities. Once patched, the image is rebuilt and rescanned to ensure all vulnerabilities have been fixed. Self-signed Root CA in Kubernetes with k3s, cert-manager and traefik In Kubernetes with k3s, cert-manager, and Traefik, you can create a self-signed root Certificate Authority (CA) to manage TLS certificates locally, useful when your cluster isn't exposed to the internet (e.g., no Let's Encrypt). The process involves setting up cert-manager to automate the issuance, renewal, and secret management of these certificates. You first create a self-signed root CA, which then signs an intermediate CA, and that intermediate CA signs leaf certificates for your services. This setup allows your services to have trusted certificates locally. ⚡TechWave: Cloud News & Analysis Google Cloud, AWS, and Azure have collaborated to introduce Kube Resource Orchestrator (kro), a Kubernetes-native tool that simplifies managing Kubernetes resources by grouping them into reusable APIs. kro allows platform and DevOps teams to define standardized deployments while hiding complexity from end users. Unlike existing tools like Helm and Kustomize, kro natively integrates with Kubernetes Custom Resource Definitions (CRDs) to create a more seamless and scalable experience. It enables organizations to create simplified interfaces for deploying applications, infrastructure, and cloud services, reducing the need for custom-built solutions. kro is open-source and still in early development, inviting community contributions. Coldplay + Microsoft AI Coldplay has teamed up with Microsoft to create an AI-powered fan experience for their new album MOON MUSiC. Using Microsoft Copilot and Azure AI, fans can generate personalized 15-second video remixes of A Film For The Future, a visual accompaniment to the album. The AI platform analyzes emotions and dynamically assembles unique clips, making each fan's experience different. Built with Azure AI Foundry, this project showcases how AI can enhance creativity by transforming audiences into co-creators. This collaboration highlights how generative AI is changing fan engagement, making music experiences more interactive and personalized. Amazon S3 Tables now support 10,000 tables per table bucket Amazon S3 Tables now allow users to create up to 10,000 tables per S3 table bucket, scaling up to 100,000 tables across 10 buckets per AWS Region per account. This expansion, available at no extra cost, enhances the ability to store and manage tabular data efficiently. S3 Tables integrate natively with Apache Iceberg, making them ideal for use with AWS Analytics services like Amazon SageMaker Lakehouse and open-source engines such as Apache Spark and Flink. This update helps businesses scale their data workloads seamlessly across all AWS Regions where S3 Tables are supported. Insights into Azure's DDoS defense During the 2024 holiday season, Azure DDoS Protection defended against evolving cyber threats, including DDoS-for-hire services, massive botnets, and politically motivated attacks. Azure mitigated up to 3,800 attacks daily, with TCP-based attacks being the most common. Notably, a Typhon botnet attack reached 125 million packets per second but was successfully blocked. Attackers increasingly use short bursts to bypass defenses, with 49% of attacks lasting under 5 minutes. To stay protected, Microsoft advises using Azure DDoS Protection with Web Application Firewall (WAF), setting up alerts, and simulating attack scenarios to enhance preparedness against future threats. DeepSeek-R1 models now available on AWS AWS has announced the availability of DeepSeek-R1 models on Amazon Bedrock Marketplace and Amazon SageMaker JumpStart, enabling users to deploy cost-effective and powerful generative AI models. Developed by Chinese AI startup DeepSeek, these models, including DeepSeek-R1-Distill, range from 1.5 to 70 billion parameters and offer 90-95% cost savings compared to similar models. Users can integrate them into Amazon Bedrock for quick deployment or Amazon SageMaker AI for advanced customization and training. Additionally, AWS Trainium and Inferentia chips provide optimized performance for these models on Amazon EC2. 🛠️HackHub: Best Tools for Cloud labring/sealos Sealos is a cloud operating system built on the Kubernetes kernel, designed to simplify managing cloud-native applications. It offers quick deployment of distributed applications and high-availability databases like MySQL, PostgreSQL, and MongoDB. apache/skywalking Apache SkyWalking is an open-source Application Performance Monitoring (APM) system designed for microservices, cloud-native, and container-based architectures. It offers end-to-end distributed tracing, service observability, and diagnostic tools, supporting various programming languages like Java, .NET, PHP, and Python. godaddy/terminus Terminus is a Node.js package that helps manage graceful shutdowns and Kubernetes health checks for HTTP applications. Terminus also provides readiness and liveness checks to inform Kubernetes about the service’s health status. alibaba/kt-connect KT-Connect is a tool that helps developers efficiently connect, redirect, and expose local applications to Kubernetes clusters for easier testing and development. stashed/stash Stash by AppsCode is a cloud-native backup and recovery solution for Kubernetes workloads, making it easier to back up and restore data like volumes and databases in dynamic Kubernetes environments. It simplifies the backup process using tools like restic and Kubernetes CSI Driver VolumeSnapshotter. 📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us. If you have any comments or feedback, just reply back to this email. Thanks for reading and have a great day! *{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Datadog Acquires QuickwitCloudPro #78: Kubernetes health checks: Best practices for configuringCloud Conversations: A Fireside Chat with Forrest Brazeal and RubrikJoin us on Jan. 28th @ 10 AM PST for a captivating fireside chat where storytelling meets cloud innovation. Forrest Brazeal—acclaimed cloud architect, author, and the creative mind behind cloud computing's most beloved cartoons—teams up with Rubrik’s Chief Business Officer, Mike Tornincasa to explore the evolving challenges of data protection in a multi-cloud world.Save Your Spot⭐MasterclassKubernetes health checks: Best practices for configuringHow to manage secrets with Azure Key Vault in Kubernetes?Self-Hosting a Container RegistryHow I tuned my CI/CD pipeline to be done in 60 secondsWhat Karpenter v1.0.0 means for Kubernetes autoscaling🔍Secret KnowledgeFive Lessons from a Minor Production IncidentMaking a Postgres Compound Index 50x FasterSQLite Index VisualizationNetworking Costs CalculatorWriting secure Go code⚡TechwaveDatadog Acquires QuickwitAzure Storage—A look back and a look forwardOpenTelemetry and Grafana Labs: what’s new and what’s next in 2025Introducing Amazon Nova foundation models: Frontier intelligence and industry leading price performanceIntroducing the next generation of Amazon SageMaker: The center for all your data, analytics, and AI🛠️HackhubGoliat Dashboard: Manage, visualize, and optimize Terraform deploymentspv-migrate:CLI tool to easily migrate Kubernetes persistent volumesGit-remote-s3:Library that enables using Amazon S3 as a git remote and LFS serverToolGit:Git Productivity ToolkitDatabend: Modern alternative to SnowflakeCheers,Shreyans SinghEditor-in-ChiefWorld’s first 16 Hour LIVE Training to become an AI-Powered human in 2025 🤖The world of AI is evolving at lightning speed, and the only way to stay relevant is to MASTER AI before it masters you.Join the World’s first 2-Day Mastermind Challenge to learn the Tools, Tactics, and Strategies to Automate Your Work Like Never Before!Best part? It is usually for $395, but the first 100 of you get in for free.Claim your FREE spot now!⭐MasterClass: Tutorials & GuidesKubernetes health checks: Best practices for configuringKubernetes health checks are essential for maintaining the reliability, performance, and availability of applications. They use probes to monitor container health and take corrective actions when necessary. The three main types of probes—Liveness, Readiness, and Startup—serve distinct purposes. Liveness probes ensure the application is running and can restart containers in case of failure. Readiness probes determine if a container is ready to handle traffic, temporarily removing it from service if it fails. Startup probes focus on verifying successful initialization for slow-starting applications. Probes can use methods like HTTP, TCP, commands, or gRPC to perform health checks.How to manage secrets with Azure Key Vault in Kubernetes?To manage secrets with Azure Key Vault in Kubernetes, you can use tools like the External Secrets Operator (ESO) and a service principal for authentication. Start by creating an Azure Key Vault, adding your sensitive data (e.g., API tokens) as secrets, and assigning the required permissions to a service principal. Install ESO on your Kubernetes cluster to synchronize secrets from Azure Key Vault to Kubernetes secrets. Then, configure a SecretStore resource in Kubernetes to connect to the Key Vault, using the service principal credentials for authentication. With this setup, applications running in Kubernetes can securely access secrets from Azure Key Vault without exposing sensitive data.Self-Hosting a Container RegistryA self-hosted container registry allows you to store and manage container images on your own infrastructure, giving you full control and independence from third-party services. It involves setting up a server with Docker, configuring a container to run the registry, securing it with user authentication (e.g., via htpasswd), and enabling HTTPS using Nginx and SSL certificates. Once configured, you can push and pull images securely from your registry. While self-hosting ensures privacy and compliance with strict regulations, it requires maintaining and securing the system yourself, making it ideal for enterprises needing tight control over their containerized workflows.How I tuned my CI/CD pipeline to be done in 60 secondsThe process of optimizing my CI/CD pipeline to run in under 60 seconds involved strategic improvements in parallelization, caching, and job refinement. Initially, my pipeline was a simple setup that took over five minutes to execute, which hampered my productivity. I split the pipeline into multiple parallel jobs, grouped similar tasks to save cost and debug time, and leveraged GitHub's caching for dependencies, linting tools, and test data to drastically reduce redundant downloads and processing. By using a Makefile for local testing, I accelerated iterations and ensured the GitHub YAML was simple and reliable. Further tuning, like combining related jobs and adding task-specific cache keys, helped balance speed and cost. These optimizations allowed me to reduce the runtime for building, testing, linting, and deploying my Golang app to under a minute, making the pipeline more efficient and developer-friendly.What Karpenter v1.0.0 means for Kubernetes autoscalingKarpenter v1.0.0 marks a significant milestone for Kubernetes autoscaling, offering a mature and stable solution for dynamic node lifecycle management. As an open-source tool designed to optimize workload placement and reduce costs, Karpenter automatically provisions and deprovisions nodes based on application demands and Kubernetes scheduling constraints. With its vendor-neutral design and integration with cloud-specific APIs like AWS, Azure, and GCP, Karpenter enhances scalability, cost-efficiency, and ease of management across diverse cloud environments. The 1.0 release ensures API stability, supports features like workload consolidation and rolling updates for node images, and enables seamless integration with other CNCF tools, empowering organizations to build intelligent and scalable cloud-native infrastructure.🔍Secret Knowledge: Learning ResourcesFive Lessons from a Minor Production IncidentA minor production incident in the AWS News platform highlighted five key lessons about software operations. First, investing in observability early paid off, as comprehensive dashboards allowed for quick identification and resolution of the issue within an hour. Second, a robust software architecture and testing regime enabled safe and confident adjustments to the system during a crisis. Third, the YAGNI principle (You Aren't Gonna Need It) has trade-offs; while simpler designs work initially, anticipating growth with safeguards like alarms could prevent issues. Fourth, bugs often travel in pairs, as one problem often uncovers or triggers another, underscoring the need for thorough debugging processes. Lastly, data lineage simplifies troubleshooting, as stored intermediate data made it easy to pinpoint and fix the root causes. These lessons underscore the importance of building resilient systems even for small-scale projects.Making a Postgres Compound Index 50x FasterOptimizing a compound index reduced query latency by 50x, showcasing the importance of index field order in PostgreSQL. Initially, a query filtering by status and event_type, and sorting by occurred_at, was slow due to an index ordered by occurred_at first. This structure forced PostgreSQL to scan millions of rows inefficiently. By reordering the index to prioritize filter fields (status, event_type) before the sort field (occurred_at), the search space narrowed significantly, enabling PostgreSQL to process only relevant subsets. This simple yet impactful adjustment improved endpoint latency from ~500ms to under 10ms, highlighting how understanding index design can drastically enhance database performance.SQLite Index VisualizationSQLite uses a B-Tree structure to organize indexes, ensuring efficient data storage and quick searches. A B-Tree consists of nodes, with each node storing cells that contain the indexed data, a row ID, and links to child nodes. The data is saved on pages, which have fixed sizes, and every index is structured hierarchically for balance and fast lookups. Using tools like sqlite3_analyzer, we can inspect indexes and visualize their layout, which includes pages, cells, and relationships. For better understanding, visualizations can be created from index data dumps, showcasing how SQLite handles different types of indexes (e.g., ASC/DESC, multi-column, and unique indexes) and optimizations through commands like VACUUM or REINDEX. This approach makes it possible to compare index designs, analyze efficiency, and explore SQLite’s inner workings.Networking Costs CalculatorThe Networking Costs Calculator is a self-hosted tool designed to estimate AWS networking costs. It includes a serverless backend that fetches updated prices for networking services using AWS Price List Query APIs, storing them in a DynamoDB table, and a ReactJS frontend hosted on S3 and CloudFront for user interaction. Users can select an AWS region, specify services, and input data transfer details to view estimated monthly costs. Deployment requires a Linux OS, NodeJS, AWS CLI, and AWS CDK, with setup guided by a provided script. The tool helps users calculate costs for features like Data Transfer, NAT Gateways, and Transit Gateway Attachments.Writing secure Go codeWriting secure Go code involves following best practices to ensure that your code is robust, secure, and performs well. Key steps include staying informed about security updates by subscribing to the Go mailing list, keeping Go versions up to date for security patches, and regularly checking for vulnerabilities using tools like go vet, staticcheck, and golangci-lint. It's also important to test code for race conditions using Go’s built-in race detector and scan for known vulnerabilities with tools like govulncheck and gosec. Regular fuzz testing and keeping dependencies updated can help prevent security issues and improve the overall quality of your code.⚡TechWave: Cloud News & AnalysisDatadog Acquires QuickwitDatadog has acquired Quickwit, an open-source, cloud-native search engine designed for fast, scalable, and cost-effective log management. This acquisition will help Datadog address the needs of organizations in regulated industries, such as finance and healthcare, that must meet strict data residency, privacy, and regulatory requirements. By integrating Quickwit, Datadog aims to provide seamless observability and real-time insights without compromising data ownership or requiring multiple logging tools. Quickwit will continue to support its open-source community with a major update under the Apache License 2.Azure Storage—A look back and a look forwardAzure Storage has played a critical role in supporting AI advancements and cloud adoption in 2024, with innovations like Azure Blob Storage enabling large-scale AI model training and Azure Elastic SAN providing cloud-native SAN capabilities. Key highlights include rapid growth in Premium SSD v2 adoption, enhanced Kubernetes support through Azure Container Storage, and improved security measures like Microsoft Defender for Storage. Looking ahead to 2025, Azure Storage aims to empower businesses with smarter data solutions, including seamless integration of unstructured data with AI services, advanced disaster recovery options, and optimized storage for mission-critical workloads, all while collaborating with key partners to drive innovation.OpenTelemetry and Grafana Labs: what’s new and what’s next in 2025OpenTelemetry, a rapidly growing open-source observability project, achieved major milestones in 2024, including support for profiling, stability for the Spring Boot starter, and updates to Semantic Conventions for databases, AI, and more. Grafana Labs actively contributed to OpenTelemetry advancements, integrating it with Prometheus and introducing tools like Grafana Alloy and Beyla for enhanced compatibility and eBPF-based auto-instrumentation. Looking ahead to 2025, the OpenTelemetry Collector is expected to reach stability with its v1 release, signaling long-term support, while new innovations like expanded eBPF capabilities and enhanced protocol support aim to simplify trace-to-profile correlation and drive broader adoption across the observability ecosystem.Introducing Amazon Nova foundation models: Frontier intelligence and industry leading price performanceAmazon Nova is Amazon's latest suite of advanced foundation models available on Amazon Bedrock, designed for both text and multimodal (text, image, and video) tasks. With models tailored for understanding (like text analysis, document processing, and multimodal reasoning) and creative content generation (producing images and videos), Nova combines top-tier intelligence with cost efficiency. Models like Nova Micro, Lite, and Pro cater to diverse business needs, from fast, low-cost tasks to complex, high-accuracy workflows, and all support extensive customization for specific industries.Introducing the next generation of Amazon SageMaker: The center for all your data, analytics, and AIAmazon SageMaker has launched its next-generation platform, integrating tools for data exploration, analytics, machine learning (ML), and generative AI into a unified environment. The revamped platform features the SageMaker Unified Studio (preview), which consolidates data and AI workflows, enabling users to process data, develop ML models, and create generative AI applications seamlessly. It introduces key capabilities like the SageMaker Lakehouse for unified data access, a visual ETL tool for data transformation, and the Amazon Bedrock IDE for building advanced generative AI solutions.🛠️HackHub: Best Tools for CloudGoliat Dashboard:The Goliat Dashboard is an open-source project built with Astro that provides an interactive interface for managing Terraform Cloud resources. It integrates seamlessly with the Terraform Cloud API to display real-time metrics and organize projects and workspaces for better resource visibility. The dashboard also supports the DigitalOcean API and plans to add Azure, AWS, and OpenAI integrations for enhanced insights. With dynamic routes and automatic updates, no additional configuration is needed after API connections.pv-migrate:pv-migrate is a command-line tool and kubectl plugin designed to simplify the migration of Kubernetes PersistentVolumeClaim (PVC) data. It addresses challenges in renaming, resizing, or moving PVCs between namespaces, clusters, or cloud providers by securely transferring data using rsync over SSH. With support for in-cluster and cross-cluster migrations, customizable manifests, and multiple migration strategies, pv-migrate enables efficient and flexible volume data handling. It supports various architectures, including arm64 and amd64, and offers shell completions for popular terminals like bash and zsh.Git-remote-s3:git-remote-s3 is a Python-based tool that enables using Amazon S3 as a Git remote and Git LFS (Large File Storage) server. It provides a seamless way to manage Git repositories and LFS files directly on S3 buckets. Users can push, pull, and manage branches in their repositories stored on S3 while ensuring encryption for security. The tool also integrates with AWS services like CodePipeline by allowing zipped repository archives for pipeline source actions. It supports concurrent users, IAM-based access control, and debug logging, making it versatile for managing versioned code or assets on AWS.ToolGit:ToolGit is a productivity toolkit for Git that extends its functionality with various custom commands and aliases to simplify and automate common Git tasks. It includes utilities for cleaning up branches, force-pulling remote changes, restoring file modes, managing branch history, and more. Easy to install, ToolGit integrates seamlessly into your workflow by adding its scripts to your PATH environment variable, enabling them as Git sub-commands. Each command comes with detailed help text for user-friendly operation, making it a practical enhancement for developers seeking efficiency in version control.Databend:Databend is an open-source cloud data warehouse built in Rust, designed as a cost-effective alternative to Snowflake. It focuses on high-speed query execution and data ingestion, supporting complex analysis of large datasets. Databend offers features such as full ACID compliance, schema flexibility, advanced indexing, and real-time data updates. It can be deployed on both cloud and on-prem environments, providing enterprise-level performance with reduced costs.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

We’re leaving Kubernetes -GitpodCloudPro #77: How to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamStop worrying about your to-do list.Zapier connects the apps you use every day, so you can focus on what matters most.Start working more efficiently - Create your free account today.Get started for free⭐MasterclassWe’re leaving Kubernetes -GitpodHow to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamCreating alerts from panels in Kubernetes Monitoring: An overlooked, powerhouse featureManaged DevOpsPoolsOptimizing Kubernetes Costs with Multi-Tenancy and Virtual Clusters🔍Secret KnowledgeI followed the official AWS Amplify guide and was charged $1,100What I wish someone told me about PostgresChoose the Right Instance Size for AWS RDSBuilding databases over a weekendMigrating billions of records: moving our active DNS database while it’s in use⚡TechwaveStreamline Kubernetes cluster management with new Amazon EKS Auto ModeOpenTelemetry for Generative AISimplify AWS governance with declarative policiesIntroducing Buy with AWSAWS Database Migration Service now automates time-intensive schema conversion tasks using generative AIAmazon DynamoDB reduces prices🛠️HackhubPkdnsis a DNS server providing self-sovereign and censorship-resistant domain names.Macosprovides a way to run macOS inside a Docker container using KVM acceleration.Pgpdfis an extension for PostgreSQL that provides a pdf data type and assorted functions.Kloudliteis an open-source platform designed to provide seamless and secure development environments for building distributed applications.OpenObserveis a cloud-native observability platform built specifically for logs, metrics, traces, analytics, and RUM designed to work at a petabyte scale.This is our final edition of CloudPro for 2024, but don’t worry—we’ll be back with more insights and updates in January 2025. In the meantime, we’ve got a little holiday treat for you!Packt has some exciting offers lined up to help you boost your tech skills and get ready for an amazing new year! It’s the perfect opportunity to relax, learn something new, and stay ahead in your field. Keep an eye out for these special holiday deals!From all of us at the Packt Newsletters team, we wish you a joyful holiday season and a fantastic start to 2025. See you next year!Cheers,Shreyans SinghEditor-in-ChiefMastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the Guide⭐MasterClass: Tutorials & GuidesWe’re leaving Kubernetes -GitpodGitpod decided to move away from Kubernetes after realizing it isn't ideal for cloud-based development environments due to their unique demands: they are highly stateful, interactive, resource-intensive, and require broad system permissions. Despite Kubernetes' strengths in scalability and orchestration for production workloads, Gitpod faced challenges with performance, security, and resource management at scale, particularly with CPU and memory usage, storage, autoscaling, and startup times. Extensive experimentation with custom solutions for these issues proved complex and limited. While Kubernetes excels for controlled, predictable application workloads, Gitpod’s experience highlighted the mismatched fit for development environments, leading to a shift toward more tailored infrastructure.How to ace (CKS 2.0) Certified Kubernetes Security Specialist ExamTo ace the Certified Kubernetes Security Specialist (CKS) 2.0 exam, start by ensuring you have a solid understanding of Kubernetes fundamentals and meet the prerequisite of obtaining the Certified Kubernetes Administrator (CKA) certification. The exam focuses on hands-on, performance-based tasks across key domains like cluster setup, hardening, system and supply chain security, and runtime monitoring. Utilize trusted study materials such as the Kubernetes documentation, platforms like KodeKloud, and mock exams from Killer.sh, which are often harder than the real exam. During the test, manage time effectively by tackling easier questions first, using aliases and shortcuts for command-line tasks, and referring to allowed documentation for efficient problem-solving.Creating alerts from panels in Kubernetes Monitoring: An overlooked, powerhouse featureGrafana Cloud's new alerting feature for Kubernetes Monitoring simplifies incident management by letting users create alerts directly from app panels. This powerful yet understated tool pulls queries from panels, lets you set thresholds, and sends notifications when they're exceeded. Ideal for tracking metrics like CPU usage, costs, and network health, it allows teams to manage infrastructure proactively.Managed DevOpsPoolsManaged DevOps Pools (MDP) simplify Azure DevOps agent management by providing a Microsoft-managed platform (PaaS) that integrates seamlessly with Azure DevOps to create scalable, secure, and customizable agent pools. Acting as a wrapper around Virtual Machine Scale Sets, MDP automates infrastructure management, allowing agents to be spun up on demand based on workload needs. It supports various image types, including preconfigured Microsoft Azure Pipeline Images, and offers private networking options for enhanced security.Optimizing Kubernetes Costs with Multi-Tenancy and Virtual ClustersManaging Kubernetes costs effectively requires innovative approaches, especially as organizations scale. Traditional methods like resource quotas, autoscaling, and cost monitoring help, but they fall short when dealing with the inefficiencies of running numerous underutilized clusters. Multi-tenancy with virtual Kubernetes clusters offers a cost-efficient solution by enabling multiple teams or applications to share a single host cluster while maintaining strong isolation and flexibility. Virtual clusters act like fully functional Kubernetes clusters within a host cluster, reducing redundancies and management fees while improving resource utilization.🔍Secret Knowledge: Learning ResourcesI followed the official AWS Amplify guide and was charged $1,100The author followed an AWS Amplify guide to integrate OpenSearch and ended up with a shocking $1,100 bill due to unexpected behaviors in the setup. Specifically, the guide’s default configurations created high-cost OpenSearch instances without making costs transparent, and resources were not properly deleted when shutting down the environment. The author identified issues like persistent OpenSearch domains and a lack of warnings about default expensive configurations. AWS refunded the charges and advised setting up budget alerts, but the problematic behavior in the guide still exists. The post cautions developers about potential pitfalls when using AWS Amplify with OpenSearch and highlights the importance of understanding AWS costs and configurations.What I wish someone told me about PostgresPostgres is a powerful but complex database system, and its vast official documentation can be overwhelming. Key tips for getting started include normalizing your data to avoid redundancy, except when performance optimizations (denormalization) are necessary. Understand SQL quirks like handling NULL as "unknown" and utilizing functions like COALESCE. Enhance the usability of psql by configuring features like pagers and shortcuts (e.g., \x for expanded view). Use indexes wisely, considering their order and suitability for different queries (e.g., prefix searches need text_pattern_ops). Be cautious with locks during operations like ALTER TABLE, as long-held locks can disrupt other processes. Embrace tools like query plans (EXPLAIN) to optimize performance, and always start with the Postgres documentation and community advice for best practices.Choose the Right Instance Size for AWS RDSTo choose the right AWS RDS instance size, start by evaluating your workload's needs in terms of CPU, memory, storage, and network bandwidth. Use AWS instance families to match these requirements, with memory-optimized instances for RAM-intensive tasks and burstable instances for cost-sensitive, sporadic workloads. Monitor key performance metrics, like CPU utilization, freeable memory, and network throughput, using AWS CloudWatch, and adjust the instance size based on consistent patterns—scale down if utilization is low and up if demands are high. Optimize performance with database tuning and continuously revisit your setup to balance cost, scalability, and performance effectively.Building databases over a weekendBuilding a database over a weekend is made feasible with tools like Apache DataFusion, which simplifies creating custom database functionalities. DataFusion provides a modular framework where you can extend or replace components like query parsing, logical and physical planning, and execution engines. By leveraging its SQL and DataFrame interfaces, you can implement custom operators, such as a streaming window operator for handling infinite data streams, by defining execution plans and integrating them into the planning pipeline. Through logical and physical optimizations, you ensure efficient query execution tailored to your use case.Migrating billions of records: moving our active DNS database while it’s in useCloudflare recently migrated its active DNS database to a new cluster to handle increasing data volumes and improve performance. Originally, DNS records were stored in a primary Postgres database alongside other services, but as Cloudflare scaled, this became increasingly problematic. The migration involved separating DNS records from other data, implementing a new gRPC API for better control, and using a Change Data Capture and Transfer Service to move data efficiently with minimal downtime. The new setup, which included better indexing and partitioning, reduced API latency and improved overall performance.⚡TechWave:CloudNews & AnalysisStreamline Kubernetes cluster management with new Amazon EKS Auto ModeWith EKS Auto Mode, AWS simplifies Kubernetes cluster management, automating compute, storage, and networking, enabling higher agility and performance while reducing operational overhead.OpenTelemetry for Generative AIOpenTelemetry is being enhanced to support observability for generative AI applications, ensuring reliable performance, cost efficiency, and safety. It introduces Semantic Conventions to standardize telemetry data across platforms and an Instrumentation Library to automate data collection, initially focusing on the OpenAI Python API. Key signals like Traces, Metrics, and Events provide insights into model behavior, usage, and interactions, aiding in debugging, optimization, and performance tuning. Developers can easily integrate this observability into applications using the provided Python library, enabling monitoring of model inputs, outputs, and operational details.Simplify AWS governance with declarative policiesAWS Declarative Policies simplify governance by enabling organizations to define and enforce cloud resource configurations centrally and at scale. Administrators can set standards, like blocking public access to VPCs or requiring specific Amazon Machine Images (AMIs), which are automatically applied across accounts, including new ones joining the organization. These policies reduce complexity by maintaining configurations even as AWS services evolve, providing actionable error messages to users for non-compliant actions. Initially supporting Amazon EC2, VPC, and EBS, declarative policies are managed via AWS Organizations and other AWS tools.Introducing Buy with AWSAWS introduces "Buy with AWS," a new feature that streamlines the procurement of cloud solutions by integrating AWS Marketplace purchasing directly into AWS Partner websites. Customers can discover, try, and purchase solutions with their AWS accounts, benefiting from simplified billing, centralized subscription management, and cost optimization tools. For example, users can start free trials or request private offers for products like Wiz or Databricks directly from Partner sites, with seamless transitions to co-branded procurement pages. Partners, in turn, can enhance their customer experience with AWS Marketplace APIs to showcase products, provide filters, and track metrics for engagement and sales.AWS Database Migration Service now automates time-intensive schema conversion tasks using generative AIAWS Database Migration Service (AWS DMS) now uses generative AI to automate up to 90% of schema conversion tasks, simplifying migrations from commercial databases to PostgreSQL. Powered by large language models hosted on Amazon Bedrock, this feature tackles complex code conversions like stored procedures and proprietary functions that traditional methods often struggle with. It reduces migration costs, accelerates timelines, and allows users to focus on optimizing their applications post-migration.Amazon DynamoDB reduces pricesAmazon DynamoDB, a serverless NoSQL database with high performance and scalability, has significantly reduced its pricing: on-demand throughput costs are now 50% lower, and global table replicated writes are up to 67% cheaper. These changes make on-demand mode—ideal for scaling serverless applications without capacity planning—the default and most cost-effective option for many workloads, even those with steady usage. Additionally, global tables now offer the same pricing for multi-Region and single-Region writes, simplifying cost management for globally distributed applications.🛠️HackHub: Best Tools for CloudPkdnsis a DNS server providing self-sovereign and censorship-resistant domain names.Macosprovides a way to run macOS inside a Docker container using KVM acceleration.Pgpdfis an extension for PostgreSQL that provides a pdf data type and assorted functions.Kloudliteis an open-source platform designed to provide seamless and secure development environments for building distributed applications.OpenObserveis a cloud-native observability platform built specifically for logs, metrics, traces, analytics, and RUM designed to work at a petabyte scale.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Google Kubernetes Engine supports 65,000-node clustersCloudPro #76: Kubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreMastering Software Deployments at the Edge: A User’s Guide to Diverting DisasterSoftware delivery to dedicated edge devices is one of the most complex challenges faced by IT professionals today. While edge deployments come with inherent complications, it’s possible to avoid the pitfalls. With this guide in hand, a little planning, and the right tools and strategies in place, you can be confident you’ll never push a faulty update at scale.Read the Guide⭐MasterclassI have asked this SSH question in every AWS interviewHow to Ace (CKAD) Certified Kubernetes Application Developer examKubernetes resource model, controller pattern and operator SDK refresherHow we avoided an outage caused by running out of IPs in EKSDeploying a Serverless REST API🔍Secret KnowledgeHow to Differentiate Manual Changes from Terraform Changes in S3 BucketManaging AWS EKS access entries with Terraform and OpenTofuUber’s billion trips migration setup30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guide⚡TechwaveKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGoogle Kubernetes Engine supports 65,000-node clustersContainer Insights with enhanced observability now available in Amazon ECSNew Amazon S3 Tables: Storage optimized for analytics workloadsGrafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source plugin🛠️HackhubNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.Cheers,Shreyans SinghEditor-in-ChiefLearn Million Dollar AI Strategies & Tools in this 3 hour AI Training for Free.This 3 hour power packed workshop that will teach you 30+ AI Tools, make you a master of prompting & talk about hacks, strategies & secrets that only the top 1% know of.By the way, here’s sneak peek into what’s inside the training:-Making money using AI-The latest AI developments, like GPT o1-Creating an AI clone of yourself, that functions exactly like YOU-10 BRAND new AI tools to automate your work & cut work time by 50%1.5 Million people are already RAVING about this hands-on Training on AI Tools. Don’t take our word for it? Attend for yourself and see.Register here⭐MasterClass: Tutorials & GuidesI have asked this SSH question in every AWS interviewIn AWS interviews, a popular and insightful question is: "You're trying to SSH into an EC2 instance, but it’s failing. How would you troubleshoot?" While it seems simple, the question evaluates a candidate's problem-solving approach, understanding of AWS infrastructure, and real-world experience. A key expectation is that candidates check security groups first since they act as firewalls controlling traffic. Surprisingly, many overlook this basic yet crucial step, diving into more complex areas instead. The question thus highlights how well candidates understand AWS fundamentals and prioritize troubleshooting steps effectively.How to Ace (CKAD) Certified Kubernetes Application Developer examThe Certified Kubernetes Application Developer (CKAD) exam is a practical certification focused on Kubernetes application deployment, maintenance, and troubleshooting. Ideal for engineers managing containerized applications in Kubernetes, it tests real-world problem-solving skills across topics like application design, deployment strategies, observability, security, and networking. The exam includes hands-on tasks performed in a live Kubernetes cluster and allows access to documentation during the test. It's considered pre-professional in difficulty, with a 66% passing score and retake opportunities. Preparation involves mastering Kubernetes CLI commands, understanding concepts like pods, deployments, and ConfigMaps, and practicing with tools like Killer.sh to simulate the exam experience.Kubernetes resource model, controller pattern and operator SDK refresher] The Resource Model uses etcd as the state store, with resources defined by objects like Kind, Group, Version, and Resource, which are mapped to API endpoints (e.g., /apis/apps/v1/deployments). Informers and SharedInformers optimize resource management by efficiently watching changes in objects, reducing API server load. Informers utilize Reflectors to fetch and cache data, Listers to retrieve objects from the cache, and Workqueues to process events like Add, Update, or Delete. Controllers act as loops that continuously reconcile the current state of resources (from their status) with the desired state (defined in their spec).How we avoided an outage caused by running out of IPs in EKSAdevinta's platform team tackled the critical issue of IP exhaustion in their EKS clusters by implementing custom networking with a secondary CIDR to allocate additional IPs, avoiding potential outages. The problem stemmed from the VPC-CNI plugin's default behavior of assigning an IP address per pod, which strained available IPs in their VPC as clusters scaled. While alternatives like switching to Cilium or enabling IPv6 were explored, the chosen solution balanced speed and reliability, enabling the team to complete their migration to EKS. By carefully testing and rolling out custom networking, the team stabilized IP usage, avoided service disruptions, and ensured seamless scaling for their multi-tenant cluster architecture.Deploying a Serverless REST APIThis guide walks you through deploying a REST API using AWS services like API Gateway, Lambda, DynamoDB, and Cognito with Terraform. The project involves creating an API that allows users to manage a list of Sicilian dishes. It starts with configuring AWS as the provider and setting up an S3 bucket to store Terraform state files. You then create an IAM role with the necessary permissions for Lambda to interact with DynamoDB. The Lambda function itself is written in Python, with methods to handle CRUD operations on the DynamoDB table based on the incoming HTTP requests. Authentication is added via Amazon Cognito to secure write operations. Finally, the API routes (GET, POST, PATCH, DELETE) are implemented to handle the dish data, including a recursive scan function to fetch all dishes from the table.🔍Secret Knowledge: Learning ResourcesHow to Differentiate Manual Changes from Terraform Changes in S3 BucketTo differentiate manual changes from Terraform changes in an S3 bucket managed by Terraform, you can use AWS CloudTrail, EventBridge, Lambda, and SNS notifications. CloudTrail logs all S3 API actions, including manual and automated changes. EventBridge filters these logs for specific events (e.g., uploads or deletions) and triggers a Lambda function. The Lambda function processes the events to exclude actions initiated by Terraform (using the IAM role or userIdentity details associated with Terraform). It sends SNS notifications only for manual changes, ensuring Terraform modifications do not trigger alerts.Managing AWS EKS access entries with Terraform and OpenTofuManaging AWS EKS access entries with Terraform and OpenTofu simplifies authentication and authorization for Kubernetes clusters by replacing the outdated aws-auth ConfigMap with a more scalable and robust EKS API. Access entries allow direct API-based management of IAM users, roles, and predefined policies, eliminating manual ConfigMap updates prone to errors. With tools like Terraform and OpenTofu, you can define access entries as Infrastructure as Code (IaC), enabling automated and secure access control at scale. This method seamlessly integrates AWS IAM for authentication with Kubernetes RBAC for authorization.Uber’s billion trips migration setupUber successfully migrated its complex trip fulfillment infrastructure to a hybrid cloud environment without downtime by employing innovative strategies. To ensure uninterrupted service for millions of global users, Uber implemented a backward compatibility layer, maintaining support for existing APIs during the transition. They also used shadow validation, mirroring requests and comparing responses between old and new systems to identify discrepancies.30 Days of CNCF projects | Day 5: What is Crossplane + demoCI/CD automation with Tekton: GitHub PR pipeline guideAutomating CI/CD with Tekton involves leveraging its Kubernetes-native framework to define pipelines for building, testing, and deploying code. Using tools like Minikube, kubectl, and Ngrok, you can set up a local Tekton environment and integrate pre-built tasks such as git-clone and kaniko for cloning repositories and building Docker images. To automate workflows triggered by GitHub pull requests, Tekton Triggers can be configured to listen for webhooks, validate events, and execute pipelines, ensuring CI/CD tasks like building, testing, and updating GitHub statuses happen seamlessly.⚡TechWave:CloudNews & AnalysisKubernetes Gateway API v1.2: WebSockets, Timeouts, Retries, and MoreGateway API v1.2 introduces significant updates and improvements to Kubernetes networking, focusing on WebSocket support, HTTPRoute timeouts, retries, and more robust infrastructure annotations. It graduates several features, like HTTPRoute timeouts and backend protocol support, to the stable Standard channel, ensuring better resilience and backward compatibility. Notably, the release brings two breaking changes: the removal of outdated v1alpha2 versions for GRPCRoute and ReferenceGrant, and a shift in .status.supportedFeatures structure for greater future flexibility.Google Kubernetes Engine supports 65,000-node clustersGoogle Kubernetes Engine (GKE) now supports clusters of up to 65,000 nodes, a scale designed to meet the growing computational demands of massive AI workloads, including training and serving trillion-parameter AI models. This enhancement allows for faster training times, larger model scalability, and flexibility in resource allocation for diverse tasks. GKE achieves this through innovations like transitioning to a Spanner-based key-value store for enhanced reliability and a revamped control plane for faster scaling and operations.Container Insights with enhanced observability now available in Amazon ECSAmazon ECS now features enhanced observability with Container Insights, helping users monitor and troubleshoot container workloads more effectively. This capability offers detailed metrics, logs, and visual dashboards to quickly identify root causes of issues, reduce detection and repair times, and improve application performance. It supports granular resource monitoring, proactive issue management, cross-account observability, and seamless integration with CloudWatch services like Application Signals and Logs.New Amazon S3 Tables: Storage optimized for analytics workloadsAmazon S3 Tables are a new storage option optimized for analytics workloads, supporting tabular data in Apache Iceberg format. This managed service provides faster query performance (up to 3x) and handles higher transactions per second (up to 10x) compared to self-managed storage. S3 Tables integrate seamlessly with query engines like Amazon Athena and Apache Spark and include features like automatic maintenance (e.g., compaction, snapshot management) and logical grouping with namespaces.Grafana 11.4 release: Introducing support for OpenSearch PPL and OpenSearch SQL in the AWS CloudWatch data source pluginGrafana 11.4 now supports OpenSearch Piped Processing Language (PPL) and OpenSearch SQL in its AWS CloudWatch data source plugin, allowing AWS users more flexibility in querying CloudWatch Logs. These new query options join the existing Logs Insights QL, enabling users to filter and aggregate logs using their preferred language without duplicating data. Features like syntax highlighting, live code completion, and sample queries enhance usability.🛠️HackHub: Best Tools for CloudNovascans your cluster for installed Helm charts and then cross-checks them against all known Helm repositories.Pglite-fusionallows you to embed an SQLite database in your PostgreSQL table. AKA multitenancy has been solved.Drasiis a data processing platform that simplifies detecting changes in data and taking immediate action.SonarIACis a static code analyzer for Infrastructure-as-Code languages such as CloudFormation and Terraform, as well as DevOps like Docker and Kubernetes.Pg_flois a CLI to move and transform data between PostgreSQL databases using Logical Replication.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

How Postgres stores data on diskCloudPro #75: A Guide to Kubernetes Network Policies⭐MasterclassA Guide to Kubernetes Network PoliciesDockerfile Instructions - ADD vs. COPYHow to add new worker node to existing Kubernetes clusterHow I Reduced Docker Image Size from 588 MB to Only 47.7 MBAmbient mesh: Can sidecar-less Istio make your application faster?🔍Secret KnowledgeOops, I Deleted the AWS Auth RolesRising Incidents on Git PlatformsHow Postgres stores data on diskHow We Integrate a New Service in Under 1 Hour for 25 ClustersEleventeen ways to delete an AWS resource🛠️HackhubApeman: AWS attack path management toolCyphernetes: A Kubernetes Query LanguageDesed:A command-line tool for complex sed scriptsKueue: Kubernetes-native Job QueueingAWS CloudFormation StarterkitCheers,Shreyans SinghEditor-in-Chief⭐MasterClass: Tutorials & GuidesA Guide to Kubernetes Network PoliciesIn Kubernetes, network policies control the traffic between pods, ensuring secure communication within the cluster. There are two main types: Layer 4 (L4) and Layer 7 (L7) policies. L4 policies manage traffic at the transport layer (e.g., TCP/UDP) based on IP addresses and ports, while L7 policies operate at the application layer (e.g., HTTP) with more fine-grained control over communication between services. L7 policies often require a service mesh like Linkerd, which adds features like mutual TLS (mTLS) for encrypted communication.Dockerfile Instructions - ADD vs. COPY`COPY` is simple and secure, only transferring files from the local build context to the image. In contrast, `ADD` offers extra functionality, such as downloading files from URLs or automatically extracting compressed archives. However, this added flexibility introduces complexity and potential security risks. Best practice recommends using `COPY` for most cases due to its straightforwardness, reserving `ADD` for situations where its unique features are necessary.How to add new worker node to existing Kubernetes clusterTo add a new worker node to an existing Kubernetes cluster, start by setting up a new Ubuntu 24.04 instance and configuring its hostname and `/etc/hosts` file. Disable swap memory, load necessary kernel modules, and install containerd as the container runtime. Add the Kubernetes APT repository, then install Kubernetes components like kubeadm, kubelet, and kubectl. On the control plane node, generate a kubeadm join command with a token. Run this command on the new worker node to join the cluster. Finally, verify the addition by checking the nodes from the control plane using `kubectl get nodes`.How I Reduced Docker Image Size from 588 MB to Only 47.7 MBTo significantly reduce a Docker image size, using multi-stage builds is key. In this case, a Flask app's image size was reduced from 588 MB to just 47.7 MB by switching to the lightweight Python 3.9-alpine image and using a multi-stage build approach. Multi-stage builds allow you to separate the build and runtime environments, keeping only essential runtime dependencies in the final image. Additionally, minimizing the number of layers by combining commands, using a `.dockerignore` file to exclude unnecessary files, and optimizing the Dockerfile structure contributed to this impressive 91.89% reduction.Ambient mesh: Can sidecar-less Istio make your application faster?Ambient mode in Istio, introduced in 2022, allows a sidecar-less architecture that can sometimes make applications faster. In traditional service meshes, adding latency is expected, but tests with ambient mode showed slightly improved performance in some cases, like the Bookinfo application's details service. This is partly because of more efficient connection handling and reduced syscalls in ambient mode, which offsets the overhead of extra hops via lightweight ztunnels.🔍Secret Knowledge: Learning ResourcesOops, I Deleted the AWS Auth RolesThe author, while managing an EKS (Elastic Kubernetes Service) cluster using Terraform, accidentally deleted the AWS authentication roles, which are crucial for accessing the cluster. This resulted in losing access to the EKS cluster. The fix involved manually restoring access by modifying the EKS API access configuration via the AWS Console, re-adding the necessary admin roles, and regenerating the `aws-auth` config map.Rising Incidents on Git PlatformsIn 2023, incidents affecting popular DevOps platforms like GitHub, Bitbucket, GitLab, and Jira increased, with issues such as RepoJacking, security vulnerabilities, and performance disruptions. GitHub saw a rise in attacks, with hackers exploiting vulnerabilities and hosting malware. Atlassian products like Bitbucket and Jira faced security flaws, with Jira experiencing a significant increase in incidents. GitLab suffered from performance issues and security breaches, including a major Proxyjacking attack.How Postgres stores data on diskPostgres stores data on disk in a well-organized, file-based structure within a directory, typically located at `/var/lib/postgresql/data`. Inside this directory, you'll find folders like `base/`, where actual database data for each database is stored, and `pg_wal/`, which holds the Write-Ahead Log (WAL) files that help recover data after crashes. Each table and database object is ultimately represented by files in these directories. PostgreSQL uses clever abstractions to manage data, such as snapshots for transactions, dynamic shared memory for handling multiple processes, and special mechanisms like tablespaces for physically separating certain data.How We Integrate a New Service in Under 1 Hour for 25 ClustersThe article describes how a team integrated a new service called Otterize across 25 clusters in under an hour, emphasizing that while the technical setup was quick, the lengthy licensing process took over four months. The integration involved automating several steps using GitOps and tools like Argo CD to avoid manual errors. Key tasks included creating an organization and environment, inviting users, integrating with Kubernetes, securely managing credentials, and deploying the setup through a script.Eleventeen ways to delete an AWS resourceOur goal is to reduce AWS costs, but the deletion methods vary widely, often leaving users frustrated. They categorize deletion patterns, from simple one-click deletes to more complex confirmations that require typing specific phrases or acknowledging consequences. Ultimately, AWS should standardize its deletion processes to improve user experience and security, and they call for more data on user behavior during these actions.🛠️HackHub: Best Tools for CloudApeman: AWS attack path management toolProject Apeman is an AWS attack path management tool that helps analyze and manage AWS security data. To set it up, you need Docker, Python, and a virtual environment. Once the system is initialized, Apeman gathers AWS account data, including authorization details and ARNs, which are then ingested into a graph database for analysis.Cyphernetes: A Kubernetes Query LanguageCyphernetes is a Cypher-inspired query language for Kubernetes, simplifying complex Kubernetes operations with intuitive, SQL-like queries. It allows developers to easily manage Kubernetes resources by expressing relationships between them, such as connecting deployments to services and ingresses.Desed:A command-line tool for complex sed scriptsDesed is a command-line tool designed to help debug and understand complex `sed` scripts. It allows users to step through their scripts, both forwards and backwards, preview how substitute commands will affect the pattern space, and set breakpoints to examine the program's state. Desed also supports hot reloading, so changes to the source code can be instantly applied without restarting the debugger.Kueue: Kubernetes-native Job QueueingKueue is a Kubernetes-native job queueing system that manages when jobs start and stop based on a variety of factors, such as priorities and resource availability. It offers features like job management with FIFO strategies, resource fair sharing, dynamic resource reclaim, and integration with popular job types like BatchJob and Kubeflow training jobs.AWS CloudFormation StarterkitAn AWS CloudFormation starterkit including CI/CD and dev tools that allow you to securely and quickly deploy CloudFormation stacks on your AWS account.📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}

Understand your Kubernetes cost drivers and the best ways to rein in spendingCloudPro #74: How Netflix solved the issue with Java 21 virtual threadsScale your scrapers with Apify’s Black Friday Boost planGet a 30% prepaid usage bonus on Apify this Black Friday.Scrape data for LLMs, machine learning, competitive intelligence, product mapping, or any AI use cases.Use ready-made scrapers or build your own.The Boost plan ends December 5 - grab it before it’s gone!Claim your bonus now⭐MasterclassDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSKubernetes and Access Management API, the new authentication in EKSHow Netflix solved the issue with Java 21 virtual threadsDoes Talos Kubernetes and Omni live up to the hype?Understand your Kubernetes cost drivers and the best ways to rein in spending🔍Secret KnowledgeHack the Cybersecurity InterviewTroubleshooting: Terminal LagMonitor these Kubernetes signals to help rightsize your fleetGetting Started with Cilium Service Mesh on Amazon EKSHow AppsFlyer migrated from Kafka to Kubernetes using Karpenter🛠️HackhubHigh-performance server for NATS.ioA collection of Bash One-Liners and terminal tricksdistributed key value NoSQL database that uses RocksDB as storage engineBuild, Share and Run Both Your Kubernetes Cluster and Distributed ApplicationsRun your deep learning workloads on Kubernetes more easily and efficientlyCheers,Shreyans SinghEditor-in-ChiefForward to a Friend⭐MasterClass: Tutorials & GuidesDeploying Databases in K8s with PostgreSQL, CloudNative-PG, and Ceph Rook on Amazon EKSThis article explains how to deploy and manage a PostgreSQL database on Kubernetes using Amazon EKS. It combines CloudNativePG, a PostgreSQL operator, with Ceph Rook, a storage orchestrator, to ensure data persistence and high availability in a Kubernetes environment. A step-by-step guide is provided for deploying and configuring these tools, including using Helm charts, setting up storage with Ceph, and monitoring the database cluster.Kubernetes and Access Management API, the new authentication in EKSIn AWS EKS (Elastic Kubernetes Service), the new authentication and access management system simplifies how users and worker nodes access Kubernetes clusters. Previously, access was managed using an "aws-auth" ConfigMap, which could be complex and prone to errors. Now, AWS introduces the EKS Access Management API, allowing users to authenticate through AWS IAM directly and authorize Kubernetes actions via Kubernetes RBAC. This approach removes the need for managing ConfigMaps manually, offers predefined EKS Access Policies, and enhances security by eliminating hidden root users. Additionally, integration with Terraform makes access control easier to implement and manage.How Netflix solved the issue with Java 21 virtual threadsIn Java 21, Netflix encountered an issue with virtual threads, which are lightweight threads designed to improve concurrency by suspending and resuming automatically. The problem involved some Netflix services using Java 21, Spring Boot 3, and Tomcat, where servers stopped processing requests due to sockets stuck in a `CLOSE_WAIT` state. Virtual threads were getting "pinned" to operating system threads while waiting to acquire locks. Since all OS threads became blocked, Tomcat couldn't process incoming requests, causing the system to hang. The underlying issue was traced to locking mechanisms in virtual threads, leading to thread contention and blocked OS threads.Does Talos Kubernetes and Omni live up to the hype?Talos Kubernetes and Sidero Omni live up to the hype by providing an intuitive and efficient way to set up and manage Kubernetes clusters. With Omni, you can easily create a Talos cluster without needing to access your virtual machines directly, making the process more streamlined. Setting up clusters, scaling nodes, and even upgrading Kubernetes versions are straightforward, with minimal manual intervention required. While there are some areas for improvement, like simplifying static IP configuration, the overall experience is highly positive.Understand your Kubernetes cost drivers and the best ways to rein in spendingTo reduce Kubernetes-related costs, it's important to monitor key cost drivers such as CPU, memory, storage, and networking. Costs are driven by resource usage and the rate at which they are consumed, so reducing unnecessary usage and optimizing resource allocation is key. Over-provisioning, idle resources, and inefficient scaling are common cost culprits. Regularly adjusting resource requests, leveraging auto-scaling tools like Horizontal Pod Autoscaler, and monitoring metrics with tools like Grafana and Prometheus can help optimize usage.🔍Secret Knowledge: Learning ResourcesHack the Cybersecurity Interview"Hack the Cybersecurity Interview, Second Edition" is a comprehensive guide designed to help individuals prepare for interviews across a wide range of cybersecurity roles. The book covers technical and behavioral interview questions for positions like cybersecurity engineer, penetration tester, and CISO, while also offering tips on personal branding, stress management, and negotiation. It provides real-world advice and industry insights, making it an essential resource for anyone looking to succeed in the competitive field of cybersecurity.Troubleshooting: Terminal LagIn this troubleshooting session, Tavis Ormandy investigates why launching the xterm terminal on his Windows machine is significantly slower compared to Fedora. He identifies that Windows applies an animation effect that delays interaction with the terminal. Through a series of tests and debugging, he discovers that the X server software (X410) adds unnecessary animation effects, which can’t be disabled directly. He uses a debugger to bypass the issue, improving the performance slightly. After further optimizations with features and caching processes, he brings the Windows terminal's performance closer to Fedora’s, significantly reducing the lag.Monitor these Kubernetes signals to help rightsize your fleetTo ensure your Kubernetes environment is both cost-efficient and sustainable, it's crucial to monitor signals like CPU, memory, disk I/O, and network utilization. Over-provisioning leads to wasted resources and high costs, while under-provisioning can degrade performance. Watch for indicators such as high resource usage, slow application performance, or low utilization to fine-tune your setup. Tools like Prometheus and Grafana, along with autoscaling, can help you dynamically adjust resources, ensuring optimal balance, reduced costs, and improved sustainability.Getting Started with Cilium Service Mesh on Amazon EKSThe blog post explains how Cilium, an open-source networking and security solution powered by eBPF, enhances network connectivity between workloads in Amazon EKS (Elastic Kubernetes Service). Cilium provides advanced networking, load balancing, encryption, and observability without the need for sidecar proxies. It integrates seamlessly with Amazon EKS to improve microservice communication, multi-cluster networking, and network policy enforcement. Cilium Service Mesh, built into Cilium, leverages eBPF and Envoy to offer high performance and low overhead for traffic management, security, and monitoring.How AppsFlyer migrated from Kafka to Kubernetes using KarpenterAppsFlyer, a global leader in mobile attribution, migrated their Kafka infrastructure to Kubernetes using Amazon EKS, simplifying management and improving performance. By switching from EC2 instances to Graviton-powered nodes, they achieved a 75% increase in throughput, 58% better write I/O, and reduced costs by 30%. AWS solutions like Strimzi Kafka Operator, Rancher’s Local Path Provisioner, and Karpenter autoscaler helped optimize local storage management and scaling. This transition cut CPU core usage in half and enhanced AppsFlyer's Kafka cluster’s scalability, efficiency, and resilience.🛠️HackHub: Best Tools for Cloudnats-io/nats-serverNATS is a high-performance, cloud-native messaging system designed for modern distributed systems, offering secure and efficient communication between digital services, devices, and systems. It supports over 40 client languages.onceupon/Bash-Oneliner"Bash-Oneliner" is a blog focusing on simple and effective bash commands for tasks like data parsing and Linux system maintenance. The commands are compatible with systems like Ubuntu, Amazon Linux, RedHat, Linux Mint, Mac, and CentOS. The blog covers topics such as variable manipulation, system management, math operations, and networking.apache/kvrocksApache Kvrocks is a distributed key-value NoSQL database that uses RocksDB as its storage engine and is compatible with the Redis protocol. It aims to reduce memory costs and increase capacity compared to Redis. Kvrocks supports key features like asynchronous replication, high availability with Redis Sentinel, and a centralized cluster management system that works with any Redis cluster client.sealerio/sealerSealer is a tool that simplifies the delivery of distributed applications by packaging a Kubernetes cluster and all application dependencies into a "ClusterImage." A ClusterImage functions similarly to a Docker image, containing everything needed to run the application, such as container images, YAML files, and Helm charts. Users can write a "Kubefile" to build this image and a "Clusterfile" to describe how to run it.kubedl-io/kubedlKubeDL is a CNCF sandbox project that simplifies running deep learning workloads on Kubernetes. It offers features like unified scheduling for training and inference, advanced optimization, and native model tracking using Kubernetes Custom Resource Definitions (CRDs).📢 If your company is interested in reaching an audience of developers and, technical professionals, and decision makers, you may want toadvertise with us.If you have any comments or feedback, just reply back to this email.Thanks for reading and have a great day!*{box-sizing:border-box}body{margin:0;padding:0}a[x-apple-data-detectors]{color:inherit!important;text-decoration:inherit!important}#MessageViewBody a{color:inherit;text-decoration:none}p{line-height:inherit}.desktop_hide,.desktop_hide table{mso-hide:all;display:none;max-height:0;overflow:hidden}.image_block img+div{display:none}sub,sup{font-size:75%;line-height:0} @media (max-width: 100%;display:block}.mobile_hide{min-height:0;max-height:0;max-width: 100%;overflow:hidden;font-size:0}.desktop_hide,.desktop_hide table{display:table!important;max-height:none!important}}