Role Overview
We are seeking a Senior Solution Architect with deep expertise in enterprise data platform engineering, modern cloud-native architectures, and high-impact AI-led engineering practices. ]. In this high-impact role, you will act as the key technical visionary and primary liaison interacting with technical counterparts at the customer level, embedded within a collaborative customer delivery team alongside dedicated DevOps and Data Engineering pods.
You will not only shape and share the technical vision but will also remain deeply ingrained in the system - driving hands-on architecture design, enforcing engineering discipline, and ensuring seamless delivery. You will bridge the gap between business strategy and highly scalable technical execution, spearheading the transition to Snowflake as our enterprise data platform, architecting end-to-end orchestration workflows leveraging Apache Airflow (with an active eye toward Airflow 3), leading large-scale multi-region pipeline refactoring, guiding data products for the Connected Planning business unit, and replacing traditional Java microservices with high-performance Python and Pandas data processing frameworks. Furthermore, you will deliver measurable business and engineering ROI through AI adoption - instituting rigorous code-review guardrails for AI-generated code, automated test generation, and proven engineering cost savings.
Experience & Qualifications
To ensure this role is grounded in current market realities while targeting elite talent capable of replacing a departing senior leader, candidates should meet the following experience profiles:
- Overall Professional Experience: Minimum of 10+ years of hands-on experience in software engineering, with 6+ years dedicated to enterprise data platform engineering and distributed systems.
- Architectural Leadership: 4+ years of experience in a Solution Architect, Lead Data Architect, or Principal Data Engineer role, with a proven track record of designing, deploying, and maintaining production-grade enterprise data and software architectures.
- Customer-Facing Technical Liaison: 3+ years of experience interacting directly with client stakeholders (Business Analysts, Product Managers, Directors, and Technical Leads) to share architectural vision and translate business requirements into technical specs.
- Core Engineering Stack: 8+ years of deep hands-on development experience in Python, Pandas, advanced SQL, and distributed data processing frameworks.
- Data Platform & Modern Warehouse (Required): 3+ years of dedicated architectural and hands-on experience with Snowflake (virtual warehouses, clustering, Snowpipe, data sharing, RBAC, and cost governance).
- Modern Data Pipelines & Streaming: 4+ years of hands-on experience designing, optimizing, and operating DAGs using Apache Airflow. Strong bonus for familiarity or migration readiness for Apache Airflow 3. (Note: Airflow orchestration is core and supersedes streaming frameworks like Kafka).
- Cloud & DevOps Ecosystems: 4+ years of experience architecting on public cloud environments (with a strong preference for AWS), utilizing containerization (Kubernetes, Docker, AWS EKS) and Infrastructure as Code (Terraform).
- Measurable AI-Led Engineering ROI (Required): 1–2+ years of demonstrated success introducing AI into the SDLC with quantifiable outcomes - including documented developer hours or dollars saved, formal review and validation processes for AI-generated code, prompt-engineering frameworks, and security/quality guardrails
Key Responsibilities
Strategic Leadership & Customer Vision
- Technical Customer Liaison: Act as the primary technical partner for the customer's engineering leads and business analysts, effectively communicating architectural vision, translating business planning requirements into technical specs, aligning technical Roadmaps, and building strong cross-functional relationships.
- Architectural Governance & Standards: Establish and champion the enterprise-wide technical standards, design templates, and architecture frameworks for data ingestion, enrichment, and publishing.
- Architecture Community of Practice (CoP): Organize, chair, and lead internal Architecture Community of Practice (CoP) sessions, fostering cross-pollination of architectural patterns and engineering best practices.
- Technology Evaluation & Strategy: Continuously evaluate emerging open-source technologies, cloud services, and AI developer tools, recommending strategic adoption where they measurably improve scalability, reliability, or cost efficiency.
- Connected Planning BU Guidance: Provide dedicated architectural oversight and technical direction for the Connected Planning business unit, whose core commercial product relies on high-throughput, mission-critical data pipelines.
Data Architecture & Engineering
- Snowflake Platform Architecture: Lead the architectural design, schema modeling, and workload optimization on Snowflake, establishing best practices for warehouse rightsizing, auto-scaling, data sharing, and role-based access control.
- Airflow Orchestration at Scale: Architect, scale, and optimize enterprise DAGs and data pipeline workflows using Apache Airflow. Lead pipeline reliability initiatives, dependency management, and modernization paths toward Airflow 3.
- Multi-Region Pipeline Re-Architecture: Lead large-scale pipeline re-architecture initiatives, including data partitioning and splitting pipelines across multiple geographical AWS cloud regions for latency, high availability, and compliance.
- Pipeline Modernization: Oversee data ingestion modernization, integrating Change Data Capture (CDC) and event-driven data flows where appropriate into PostgreSQL and Snowflake.
- Data Governance & Quality: Enforce enterprise-wide schema cataloging and data contracts. Ensure data quality, security, compliance, and governance standards are built natively into all pipelines.
Cloud-Native & AI-Led SDLC
- Python Data Engineering Architecture: Design robust, scalable, and maintainable Python- and Pandas-based data processing architectures, ensuring seamless integration with data pipelines, APIs, and AWS EKS cloud infrastructure.
- Measurable AI Productivity & Governance: Drive the strategic adoption of AI tools with formal guardrails: implement strict code-review workflows for AI-generated code, prompt-engineering frameworks, and automated test-generation tooling to maximize delivery speed while preventing technical debt.
- Quantifiable ROI Tracking: Track, quantify, and report tangible engineering impact from AI adoption (e.g., dollars saved in engineering capacity, reduction in developer turnaround time, automated test coverage improvements).
- Spec-Driven Development: Enforce a disciplined development model: Requirements → Specifications → User Stories → Test Cases → Code → CI/CD Deployment.
Delivery Discipline & Operations
- Engineering Rigor: Push engineering teams, project managers, and product owners toward stronger delivery discipline by implementing strict design validation, rigorous code reviews, automated unit/integration testing, and release governance in close collaboration with embedded DevOps and Data Engineering pods.
- Platform Reliability & Production Escalations: Proactively troubleshoot and resolve complex, high-priority production incidents. Support ongoing capacity planning and cloud infrastructure cost optimization (specifically Snowflake credit consumption, AWS EKS compute clusters, and PostgreSQL).
- Agile Collaboration: Lead and participate in agile ceremonies (scrums, reviews, retrospectives), providing clarity on technical blockers, achievements, and technical debt.
Technical Skills Matrix
To ensure clear alignment between hiring managers and the Talent Acquisition team, candidate technical depth is evaluated across Hands-on Depth vs. Architectural Oversight:
Category
Expert Level (Hands-on & Deep Code Rigor)
Architectural Awareness (Design, Integration & Oversight)
Data Engineering & Languages
Python, Pandas, Advanced SQL, PySpark/data processing libraries
Data Modeling (Dimensional, Data Vault), RESTful Data APIs, Data Contracts
Data Platform & Warehousing
Snowflake (Warehouses, Snowpipe, Clustering, Streams/Tasks, RBAC, Cost Management)
PostgreSQL Optimization, Data Lakes (Parquet/Iceberg), Vector Datastores
Pipeline Orchestration
Apache Airflow (DAG Design, Custom Operators, Airflow 2.x, Airflow 3 Awareness/Readiness)
Event Streaming / Messaging (Kafka, Kafka Connect/CDC), Celery/Kubernetes Executors
Cloud & DevOps Infrastructure
AWS (S3, IAM, VPC, EC2), AWS EKS (Kubernetes), Docker, CI/CD Automation
Terraform / IaC, Helm, Prometheus, Grafana, CloudWatch
AI Integration & FinOps
AI-Assisted Testing, AI Code Review Guardrails, Developer Cost/Productivity Metrics, Snowflake Credit Optimization
Prompt Engineering Frameworks, LLM Agents/RAG architectures, AI Security/IP Compliance, FinOps
Education & Certifications (Preferred)
- Education: Bachelor’s or Master’s degree in Computer Science, Software Engineering, Information Systems, or a related technical discipline
- Certifications:
- Snowflake Certified: SnowPro Core or SnowPro Advanced Architect (highly valued)
- AWS Certified Solutions Architect – Professional (highly preferred)
- Astronomer Certified: Apache Airflow Fundamentals or DAG Authoring / Data Engineering Certifications
- Confluent Certified Developer for Apache Kafka or equivalent Data Engineering certifications.