Description
We're looking for a Senior Software Engineer to join our team, focusing on a next-generation multi-cloud FinOps platform. You'll own architecture and critical components end-to-end, raising engineering standards and partnering with FinOps practitioners and product leadership to ship measurable impact.
What Success Looks Like (Performance Objectives)
- Multi-Cloud Ingestion (E.g. AWS / Azure / GCP)
Within 180 days, we will deliver ingestion services that process high-volume billing data with clear SLAs.
Key outcomes
- AWS ingestion supports CUR files in S3, Cost Explorer APIs, Cost Categories, RI/Savings Plans coverage.
- Azure ingestion supports Cost Management exports/APIs, EA/billing constructs where applicable.
- GCP ingestion supports BigQuery billing export tables and relevant cost APIs.
- Ingestion can handle TB-scale datasets with partitioning/compaction strategies and repeatable backfills.
- Data quality checks and lineage are built-in (schema drift detection, late-arriving data handling, idempotency).
Success measures
- ≥ 99% ingestion job success rate (or agreed SLA)
- Known error budget and automated retries/alerts
- Backfills complete within agreed time windows
- Cost data freshness targets are met (e.g., daily/hourly depending on provider constraints)
- Unified Cost Data Model & Normalization
Within 6-9 months, we continue the story with the ability to deliver a normalized, cross-cloud cost model enabling allocation and analytics consistency.
Key outcomes
- A unified schema that resolves cross-cloud differences (accounts/subscriptions/projects, resource identity, usage types).
- Support for allocation via tags/labels, custom dimensions, shared cost modeling, and business mappings.
- A detailed semantic layer (definitions for amortized vs. blended, commitment allocation, etc.).
- A versioned approach for schema evolution and backward compatibility.
Success measures
- Allocation accuracy validated with FinOps customers
- Clear “source of truth” definitions and reconciliation process
- Schema changes do not break downstream dashboards/queries
- High-Performance Analytics Engine (Interactive + Explainable)
Within 9–12 months, we will enable fast queries for “where did the spend go?”
Key outcomes
- Analytics pipelines optimized for interactive exploration (e.g., drill-downs, group-by dimensions, cost drivers).
- Support for anomaly detection workflows (rules-based and/or ML-assisted is a plus).
- Query patterns built for performance at scale (partitioning, clustering, materialization strategy).
- Integration with warehouse/Lakehouse depending on your stack.
Success measures
- Key dashboards load within target latency (e.g., <2–5 seconds for common views)
- Defined SLAs for compute cost, query performance, and data freshness
- Reduction in support critical issues tied to data discrepancies
- Product-Grade Full Stack Delivery (Backend + UI)
Within 90–180 days, end-user capabilities will be shipped to accelerate FinOps workflows.
Key outcomes
- Backend services in Python, expose stable APIs for analytics and allocations.
- TypeScript UI (React/Next.js) supports high-performance tables, explorers, and filters.
- Reusable API patterns and consistent contract/versioning.
- Thoughtful UX for FinOps use cases: allocation review, cost driver exploration, commitment coverage insights.
Success measures
- Feature adoption and repeat usage in pilot teams
- Reduction in manual reporting effort for FinOps teams
- Performance benchmarks meet agreed targets
- Engineering Standards, Reliability & Cost Efficiency!
Ongoing, we will raise the platform bar across security, observability, and operational perfection.
Key outcomes
- Standards for microservices, API architecture, data modeling, distributed system patterns.
- Observability: tracing/logging/metrics, SLOs, alerting, on-call readiness where applicable.
- Security and governance appropriate for billing and usage data (RBAC, audit, encryption, least privilege access).
- Cost-aware engineering: FinOps principles applied to the platform itself.
Success measures
- Production incidents reduced over time, with blameless postmortems and preventive action
- Infrastructure costs supervised and optimized with clear ownership
- Engineering guidelines accepted by the team and reflected in PR quality
Your Core Responsibilities (What You’ll Do)
- Architect and build ingestion, normalization, and analytics components for multi-cloud billing data.
- Lead reviews and establish engineering practices for reliability, scalability, and cost efficiency.
- Deliver high-quality backend services and modern UI experiences for FinOps users.
- Partner with FinOps Reporting and Analytics, Product, and Cloud Engineering to build roadmap and ensure correctness.
- Mentor engineers through code reviews, pairing, design guidance, and technical standards.
The Environment (Representative Tech Stack)
- Languages: Python, TypeScript
- Backend: FastAPI / Flask / Django; microservices; event-driven patterns.
- Data/Compute: Spark/PySpark/Dask; serverless ETL (Glue/Synapse/Dataflow)
- Storage/Warehouse: Snowflake, BigQuery, Redshift, Synapse; Lakehouse patterns
- Infra: Docker/Kubernetes; IaC (Terraform/CDK/Pulumi)
- Observability: OpenTelemetry-style tracing, metrics/logs, SLOs
- Clouds: Cloud billing constructs and APIs
Candidate Profile (Evidence of Comparable Accomplishments) You’re likely a strong fit if you’ve done several of these:
- Develop production ingestion pipelines for large datasets (TB-scale or high-frequency) with quality controls and SLAs.
- Crafted multi-tenant or multi-account analytics platforms with consistent schemas and performance constraints.
- Implemented cost/usage analytics or similar domains (telemetry, observability, billing, metering, FinOps, data platforms).
- Shipped full stack features: APIs + UI that users rely on daily (not just prototypes).
- Led architecture decisions and influenced standards across a team (architecture design documents, reviews, migration strategy).
- Worked with CUR/exports, commitment models, allocation, tagging/labels, reconciliation.
Critical Proficiencies (What “Extraordinary” Looks Like in the Role)
- Systems thinking: sees end-to-end flow and optimizes for reliability + cost + usability.
- Data correctness perspective: makes data credible, explainable, and defensible.
- Product engineering: builds for users, not just infrastructure.
- Pragmatic architecture: chooses simple, scalable approaches; avoids over-engineering.
- Influence & mentorship: improves via coaching, standards, and streamlined communication.