Senior Data Engineer
Há 1 mês
Job Title: Senior Data Engineer
Company Description: Devexperts works with respected financial institutions, delivering products and solutions for retail and brokerage houses, exchanges, and firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.
Job Description:
We are looking for a Senior Data Engineer with a Java/Scala/Python background to join the Research & Development Team.
Our current project is for a Top-5 US retail broker (by the number of users). The project is devoted to trading experience, finance reports, and risk management.
You will join a functional team that excels in getting features done from zero to production.
Responsibilities:
1. Develop Data Pipeline:
Design, develop, and maintain robust data pipelines using Java within AWS infrastructure.
Implement scalable solutions for data analysis and transformation using Apache Spark and PySpark.
Utilise Airflow for efficient workflow orchestration in complex data processing tasks.
Ensure fast and interactive querying capabilities through the use of Presto.
2. Manage Infrastructure:
Containerise applications using Docker for streamlined deployment and scaling.
Orchestrate and manage containers effectively with Kubernetes in production environments.
Implement infrastructure as code using Terraform for provisioning and managing AWS resources.
3. Collaborate and Communicate:
Collaborate with functional teams to understand data requirements and architect scalable solutions aligned with business goals.
Ensure data quality and reliability through robust testing methodologies and monitoring solutions.
Stay updated with emerging technologies and industry trends to continuously enhance the data engineering ecosystem.
Qualifications:
Must-have skills:
1. Education and Experience:
Bachelor's degree in Computer Science, Engineering, or related field.
Minimum 5 years of experience in Java/Scala/Python development, emphasising object-oriented principles.
2. Technical Proficiency:
Proficiency in Apache Spark or PySpark for large-scale data processing.
Experience with Airflow for workflow orchestration in production environments.
Familiarity with Docker for containerisation and Kubernetes for container orchestration.
Experience managing AWS services such as S3, EMR, Glue, Athena, and Redshift.
Strong background in SQL and relational databases.
3. Communication Skills:
Excellent English language communication skills, both verbal and written.
Ability to collaborate effectively with technical and non-technical stakeholders.
Nice-to-have skills:
Experience with streaming platforms such as Kafka for real-time data processing.
Knowledge of Terraform for infrastructure as code implementation in AWS environments.
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Park Senior Villas Chandler Tempo inteiroAbout the RoleWe are seeking a highly skilled Senior Dataops Engineer to join our team at Park Senior Villas Chandler. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines. Your expertise in automating data management processes in Azure Data Factory, AWS Glue or...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Devexperts Llc Tempo inteiroSenior Data EngineerAt Devexperts LLC, we're seeking a highly skilled Senior Data Engineer to join our Research & Development Team. Our current project involves working with a top US retail broker, focusing on trading experience, finance reports, and risk management.Key Responsibilities:Design, develop, and maintain robust data pipelines using Java within...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Devexperts Llc Tempo inteiroJob Title: Senior Data EngineerAbout the Role:Devexperts Llc is seeking a highly skilled Senior Data Engineer to join our Research & Development Team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining robust data pipelines using Java within AWS infrastructure. You will also implement scalable solutions for data...
-
Senior Data Engineer
3 semanas atrás
Porto, Porto, Portugal Aitopics Tempo inteiroSenior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Planet. As a Senior Data Engineer, you will play a key role in building and maintaining our data infrastructure, with a focus on Snowflake, Fivetran, and dbt.Key Responsibilities:Design and develop efficient and scalable data pipelines using Fivetran for data...
-
Senior Data Engineer
2 semanas atrás
Porto, Porto, Portugal Swordhealth Tempo inteiroJob SummaryWe are seeking a talented Senior Data Engineer to join our team at Swordhealth. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining ETL/ELT data pipelines that ingest data from multiple sources with different technologies.Main Responsibilities• Design, build, and maintain ETL/ELT data pipelines that ingest...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Kcs It Tempo inteiroWe're seeking a talented Data Engineer to join our team at KCS IT. As a senior member of our team, you'll be responsible for designing, developing, and maintaining large-scale data systems using PySpark, SQL, and Airflow. Our ideal candidate has a strong background in data engineering, with expertise in PySpark and SQL. Experience with Airflow and GCP is a...
-
Senior Data Engineer Officer
3 semanas atrás
Porto, Porto, Portugal Match Profiler Tempo inteiroJob Title: Senior Data Engineer OfficerAbout Us: Match Profiler is a consulting firm with multidisciplinary competences in IT, empowering clients through optimized solutions.Job Summary: We are seeking a Senior Data Engineer Officer to join our team. This role requires a deep understanding of software development cycles, data manipulation, and complex...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing data warehouse solutions on Snowflake, and developing and...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data Engineer - Cloud Data ArchitectAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team in Portugal. As a Senior Data Engineer, you will be responsible for designing, implementing, and operating cloud-based data architectures on Amazon Web Services (AWS).Key Responsibilities:Design and implement cloud-based...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Phiture Tempo inteiroAbout the Role:We are seeking an exceptional Senior Data Engineer to join our team at Phiture. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement complex data pipelines using Azure Data Factory and DatabricksDevelop and...
-
Senior Data Engineer Position
4 semanas atrás
Porto, Porto, Portugal Kcs It Tempo inteiroAbout the RoleWe're seeking a Senior Data Engineer with 5+ years of experience to join our team in Porto, Portugal. As a Senior Data Engineer, you will be responsible for designing and developing end-to-end data pipelines using Alteryx, ETL processes, and Data Mesh concepts. You will also have expertise in programming languages such as Python, Java, and...
-
Senior Data Platform Engineer
2 semanas atrás
Porto, Porto, Portugal Kuehne+Nagel Tempo inteiroJob Role: Senior Data Platform EngineerAbout Us: Kuehne+Nagel is a leading global logistics company that is shaping the future of logistics and IT.Your Role: As a Senior Data Platform Engineer, you will be part of our central data platform team, designing and implementing our data mesh approach on a cloud transformation program.Responsibilities:Design and...
-
Senior Data Analytics Engineer
Há 1 mês
Porto, Porto, Portugal Protegrity Tempo inteiroUnlock the Power of Data AnalyticsProtegrity, a leading provider of data protection solutions, is seeking a talented Senior Data Analytics Engineer to join our Business Intelligence Team. This role is a remote opportunity that offers the chance to make a significant impact in a company that prioritizes data-driven decision-making.As a Senior Data Analytics...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Capgemini Tempo inteiroAbout the RoleAs a Senior Engineer at Capgemini, you will be part of a dynamic team that designs and implements distributed data processing solutions for various business cases. Your expertise in Python and experience with modern development practices will be crucial in creating scalable and efficient solutions.Your ResponsibilitiesInvestigate and resolve...
-
Senior Data Analytics Engineer
Há 1 mês
Porto, Porto, Portugal Protegrity Tempo inteiroJob SummaryProtegrity is seeking a highly skilled Senior Data Analytics Engineer to join our Business Intelligence Team. As a Senior Data Analytics Engineer, you will be responsible for strengthening Protegrity's data analytics and information reporting capabilities, using modern techniques and tools to ensure efficient scalability.Key...
-
Senior Data Engineer
3 semanas atrás
Porto, Porto, Portugal Capgemini Tempo inteiroAt Capgemini Engineering, we're seeking a Senior Data Engineer to build distributed data processing solutions and highly loaded database solutions for various business cases. As part of our self-organized team of experienced data engineers, you'll work in a challenging, innovative environment for our client, creating the foundation for decision-making at a...
-
Senior Data Engineer Transformation Lead
1 semana atrás
Porto, Porto, Portugal Five9 Tempo inteiroAbout Five9Five9 is a leading provider of cloud contact center solutions, dedicated to helping businesses deliver exceptional customer experiences.Job Description:Data Engineer (Senior) Job SummaryWe are seeking an experienced Data Engineer to join our team. As a senior data engineer, you will be responsible for designing, building, and maintaining...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal WE ARE META Tempo inteiroAt WE ARE META, we're seeking a skilled Senior Data Engineer to join our team. As a Rhino, you'll be part of a dynamic group that focuses on finding the perfect match between our experts and clients.Why become a Rhino?Expand your network: As a consultant at We Are META, you'll have access to a network of national and international partners across diverse...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Euronext Tempo inteiroJob DescriptionEuronext is seeking a talented Senior Data Engineer with expertise in AWS and a strong background in solution architecture, database management, and data engineering. The ideal candidate will have extensive experience with AWS services such as Lambda, Glue, Step Functions, and CloudFormation, along with proficiency in Python, SQL, and database...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Multivison, Lda Tempo inteiroJob Requirements: We are seeking a highly skilled Senior Data Engineer with expertise in Apache Flink and a strong background in data processing. The ideal candidate will have at least 5 years of experience and a C1 level of English. Key Responsibilities: Optimize the use of Apache Flink to improve the efficiency of our client's systems. Increase client...