Senior Data Engineer

4 meses atrás

Porto, Portugal Devexperts Tempo inteiro

Company Description

Devexperts works with respected financial institutions, delivering products and tailor-made solutions for retail and brokerage houses, exchanges, and buy-side firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services. 

Job Description

We are looking for a  Senior Data Engineer with a Java / Scala / Python background  to join the Research & Development Team. 

Our current project is for a Top-5 US retail broker (by the number of users). The project is devoted to trading experience, finance reports, and risk management.

You will join cross-functional team that excels in getting features done from zero to production.

We expect the Senior Data Engineer to: 

1. Develop Data Pipeline:

Design, develop, and maintain robust data pipelines using Java within AWS infrastructure, Implement scalable solutions for data analysis and transformation using Apache Spark and PySpark, Utilise Airflow for efficient workflow orchestration in complex data processing tasks, Ensure fast and interactive querying capabilities through the use of Presto.

2. Manage Infrastructure:

Containerise applications using Docker for streamlined deployment and scaling, Orchestrate and manage containers effectively with Kubernetes in production environments, Implement infrastructure as code using Terraform for provisioning and managing AWS resources.

3. Collaborate and Communicate:

Collaborate with cross-functional teams to understand data requirements and architect scalable solutions aligned with business goals, Ensure data quality and reliability through robust testing methodologies and monitoring solutions, Stay updated with emerging technologies and industry trends to continuously enhance the data engineering ecosystem.

Qualifications

Must-have skills: 

1. Education and Experience:

Bachelor's degree in Computer Science, Engineering, or related field, Minimum 5 years of hands-on experience in Java / Scala / Python development, emphasising object-oriented principles.

2. Technical Proficiency:

Proficiency in Apache Spark or PySpark for large-scale data processing, Experience with Airflow for workflow orchestration in production environments, Familiarity with Docker for containerisation and Kubernetes for container orchestration, Experience managing AWS services such as S3, EMR, Glue, Athena, and Redshift, Strong background in SQL and relational databases.

3. Communication Skills:

Excellent English language communication skills, both verbal and written, Ability to collaborate effectively with technical and non-technical stakeholders.

Nice-to-have skills: 

Experience with streaming platforms such as Kafka for real-time data processing, Knowledge of Terraform for infrastructure as code implementation in AWS environments.

  • Senior Data Engineer

    4 semanas atrás

    Porto, Porto, Portugal Aitopics Tempo inteiro

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Planet. As a Senior Data Engineer, you will play a key role in building and maintaining our data infrastructure, with a focus on Snowflake, Fivetran, and dbt.Key Responsibilities:Design and develop efficient and scalable data pipelines using Fivetran for data...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Swordhealth Tempo inteiro

    Job SummaryWe are seeking a talented Senior Data Engineer to join our team at Swordhealth. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining ETL/ELT data pipelines that ingest data from multiple sources with different technologies.Main Responsibilities• Design, build, and maintain ETL/ELT data pipelines that ingest...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Kcs It Tempo inteiro

    We're seeking a talented Data Engineer to join our team at KCS IT. As a senior member of our team, you'll be responsible for designing, developing, and maintaining large-scale data systems using PySpark, SQL, and Airflow. Our ideal candidate has a strong background in data engineering, with expertise in PySpark and SQL. Experience with Airflow and GCP is a...

  • Senior Data Engineer Officer

    4 semanas atrás

    Porto, Porto, Portugal Match Profiler Tempo inteiro

    Job Title: Senior Data Engineer OfficerAbout Us: Match Profiler is a consulting firm with multidisciplinary competences in IT, empowering clients through optimized solutions.Job Summary: We are seeking a Senior Data Engineer Officer to join our team. This role requires a deep understanding of software development cycles, data manipulation, and complex...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing data warehouse solutions on Snowflake, and developing and...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data Engineer - Cloud Data ArchitectAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team in Portugal. As a Senior Data Engineer, you will be responsible for designing, implementing, and operating cloud-based data architectures on Amazon Web Services (AWS).Key Responsibilities:Design and implement cloud-based...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Phiture Tempo inteiro

    About the Role:We are seeking an exceptional Senior Data Engineer to join our team at Phiture. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement complex data pipelines using Azure Data Factory and DatabricksDevelop and...

  • Senior Data Platform Engineer

    3 semanas atrás

    Porto, Porto, Portugal Kuehne+Nagel Tempo inteiro

    Job Role: Senior Data Platform EngineerAbout Us: Kuehne+Nagel is a leading global logistics company that is shaping the future of logistics and IT.Your Role: As a Senior Data Platform Engineer, you will be part of our central data platform team, designing and implementing our data mesh approach on a cloud transformation program.Responsibilities:Design and...

  • Senior Data Engineer

    4 meses atrás

    Porto, Portugal Devexperts Llc Tempo inteiro

    Devexperts works with respected financial institutions, delivering products and tailor-made solutions for retail and brokerage houses, exchanges, and buy-side firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services. Job DescriptionWe are looking for...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerCompany Description: Devexperts works with respected financial institutions, delivering products and solutions for retail and brokerage houses, exchanges, and firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.Job...

  • Senior Data Engineer

    2 meses atrás

    Porto, Portugal Jscrambler Tempo inteiro

    Company DescriptionAt Jscrambler, we're on an exciting journey of rapid growth, fueled by the success of our innovative web security solutions. Our cutting-edge cybersecurity technology is trusted by Fortune 500 companies and leading global organizations to safeguard their online presence and applications. (Check out our website for impressive success...

  • Senior Data Engineer

    4 meses atrás

    Porto, Portugal Devexperts Llc Tempo inteiro

    Devexperts works with respected financial institutions, delivering products and tailor-made solutions for retail and brokerage houses, exchanges, and buy-side firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.Job Description We are looking for...

  • Senior Data Engineer

    Há 1 mês

    Porto, Portugal We Are Meta Tempo inteiro

    Seeking a Senior Data Engineer: Rhino, are you there?At WE ARE META, we focus on finding the perfect match between our Rhinos and our clients.Why join us?Expand your network: As a consultant at We Are META, you'll have access to a network of national and international partners across diverse sectors of the tech industry.Enjoy our perks: When you join the...

  • Senior Data Engineer

    2 semanas atrás

    Porto, Portugal Tui Group Tempo inteiro

    TUI Group is the world's number one integrated tourism business. Airline Data Solutions is a global team within TUI technology responsible for effective delivery of IT capabilities that support our global airline. We are a multi-disciplinary team of experts across Architecture, Engineering, DevOps and Agile Delivery providing services across the UK, Ireland,...

  • Senior Data Engineer

    2 meses atrás

    Porto, Portugal Tn Portugal Tempo inteiro

    col-wideJob Description:Company DescriptionDevexperts works with respected financial institutions, delivering products and- made solutions for retail and brokerage houses, exchanges, and- side firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.Job...

  • Senior Data Engineer

    4 semanas atrás

    Porto, Porto, Portugal Capgemini Tempo inteiro

    At Capgemini Engineering, we're seeking a Senior Data Engineer to build distributed data processing solutions and highly loaded database solutions for various business cases. As part of our self-organized team of experienced data engineers, you'll work in a challenging, innovative environment for our client, creating the foundation for decision-making at a...

  • Senior Data Engineer

    Há 1 mês

    Porto, Portugal Phiture Tempo inteiro

    Seeking a Senior Data Engineer: Rhino, are you there?At WE ARE META, we focus on finding the perfect match between our Rhinos and our clients.Why join us? Expand your network: As a consultant at We Are META, you'll have access to a network of national and international partners across diverse sectors of the tech industry.Enjoy our perks: When you join the...

  • Senior Data Engineer Transformation Lead

    2 semanas atrás

    Porto, Porto, Portugal Five9 Tempo inteiro

    About Five9Five9 is a leading provider of cloud contact center solutions, dedicated to helping businesses deliver exceptional customer experiences.Job Description:Data Engineer (Senior) Job SummaryWe are seeking an experienced Data Engineer to join our team. As a senior data engineer, you will be responsible for designing, building, and maintaining...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal Euronext Tempo inteiro

    Job DescriptionEuronext is seeking a talented Senior Data Engineer with expertise in AWS and a strong background in solution architecture, database management, and data engineering. The ideal candidate will have extensive experience with AWS services such as Lambda, Glue, Step Functions, and CloudFormation, along with proficiency in Python, SQL, and database...

  • Senior Data Engineer

    Há 1 mês

    Porto, Porto, Portugal WE ARE META Tempo inteiro

    At WE ARE META, we're seeking a skilled Senior Data Engineer to join our team. As a Rhino, you'll be part of a dynamic group that focuses on finding the perfect match between our experts and clients.Why become a Rhino?Expand your network: As a consultant at We Are META, you'll have access to a network of national and international partners across diverse...