Data Engineer

We are looking for a Senior GCP Data Engineer to support a strategic data transformation and cloud migration initiative. In this role, you will help migrate databases, ETL processes, and analytical workloads from SQL Server to Google Cloud Platform, designing a modern, scalable, and cost-efficient data ecosystem powered by BigQuery, Cloud Composer, and Python. You will work closely with Database Administrators, Data Engineers, Analysts, Data Scientists, and business stakeholders to understand the existing environment, rebuild critical data flows, and ensure a smooth transition to the new cloud platform. This is an excellent opportunity for someone who combines strong data engineering expertise with hands-on cloud migration experience and enjoys solving complex data, performance, and integration challenges. As a Senior GCP Data Engineer, you will:
- Contribute to the migration of data and ETL processes from SQL Server to GCP.
- Design and build scalable data pipelines using modern cloud technologies.
- Develop optimized data models and queries for BigQuery.
- Automate migration, transformation, validation, and integration activities using Python.
- Guarantee data quality, integrity, performance, and operational reliability throughout the migration.
- Help establish engineering standards and best practices for the new GCP data platform.
- Support teams and end users during the transition to the cloud environment. Data & Database Migration
- Contribute to the design and execution of the migration strategy from SQL Server to Google Cloud Platform.
- Analyze existing databases, ETL workflows, stored procedures, dependencies, and data structures.
- Develop migration scripts and pipelines to transfer and transform data into BigQuery.
- Rebuild or adapt existing SQL Server processes for a cloud-native data architecture.
- Support migration planning, technical assessments, testing, reconciliation, and production deployment.
- Ensure data consistency and integrity before, during, and after migration.
- Identify migration risks, technical dependencies, and potential performance issues.
- Design, develop, and maintain robust ETL and ELT pipelines on Google Cloud Platform.
- Replace or modernize existing data flows using scalable cloud services.
- Build reusable components and standardized pipeline patterns.
- Ensure pipelines are reliable, maintainable, observable, and performance-oriented.
- Implement appropriate retry, recovery, logging, and error-handling mechanisms.
- Support both batch processing and data integration use cases. Workflow Orchestration
- Create, schedule, monitor, and maintain workflows using Google Cloud Composer.
- Design and manage DAGs for migration, transformation, and integration processes.
- Define dependencies, execution schedules, retries, alerts, and failure-handling processes.
- Troubleshoot orchestration issues and optimize workflow execution.
- Ensure reliable operationalization of pipelines across development, testing, and production environments. BigQuery Engineering & Optimization
- Design and implement scalable data structures in BigQuery.
- Develop and optimize complex SQL queries for analytical and transformation workloads.
- Adapt T-SQL logic and existing SQL Server processes to BigQuery SQL.
- Apply BigQuery optimization techniques, including:Partitioning
- Query optimization
- Storage optimization
- Monitor query performance and resource consumption.
- Contribute to the design of cloud data models aligned with business and analytical requirements. Python Development & Automation
- Develop Python scripts for:Data migration
- Data transformation
- Integration with GCP services and APIs
- Build reusable and maintainable Python components.
- Automate repetitive migration and operational activities.
- Support the development of validation and monitoring utilities.
- Apply software engineering best practices, including code reviews, version control, testing, and documentation. Data Quality & Validation
- Implement data quality controls throughout the migration lifecycle.
- Develop automated reconciliation processes between source and target systems.
- Validate data completeness, accuracy, consistency, and integrity.
- Define tests for migrated tables, transformations, pipelines, and business rules.
- Investigate data discrepancies and coordinate their resolution.
- Ensure that the new platform produces reliable and trusted data for its users. Monitoring, Support & Troubleshooting
- Monitor data pipelines, scheduled workflows, and production processes.
- Investigate and resolve pipeline failures, data issues, and performance bottlenecks.
- Support debugging and incident resolution during the migration and after production deployment.
- Perform root cause analysis and implement sustainable corrective measures.
- Improve platform observability, alerts, operational procedures, and support documentation. Collaboration & Stakeho