Senior Data Engineer

3 meses atrás

Porto, Portugal Devexperts Tempo inteiro

Company Description

Devexperts works with respected financial institutions, delivering products and tailor-made solutions for retail and brokerage houses, exchanges, and buy-side firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services. 

Job Description

We are looking for a  Senior Data Engineer with a Java / Scala / Python background  to join the Research & Development Team. 

Our current project is for a Top-5 US retail broker (by the number of users). The project is devoted to trading experience, finance reports, and risk management.

You will join cross-functional team that excels in getting features done from zero to production.

We expect the Senior Data Engineer to: 

1. Develop Data Pipeline:

Design, develop, and maintain robust data pipelines using Java within AWS infrastructure, Implement scalable solutions for data analysis and transformation using Apache Spark and PySpark, Utilise Airflow for efficient workflow orchestration in complex data processing tasks, Ensure fast and interactive querying capabilities through the use of Presto.

2. Manage Infrastructure:

Containerise applications using Docker for streamlined deployment and scaling, Orchestrate and manage containers effectively with Kubernetes in production environments, Implement infrastructure as code using Terraform for provisioning and managing AWS resources.

3. Collaborate and Communicate:

Collaborate with cross-functional teams to understand data requirements and architect scalable solutions aligned with business goals, Ensure data quality and reliability through robust testing methodologies and monitoring solutions, Stay updated with emerging technologies and industry trends to continuously enhance the data engineering ecosystem.

Qualifications

Must-have skills: 

1. Education and Experience:

Bachelor's degree in Computer Science, Engineering, or related field, Minimum 5 years of hands-on experience in Java / Scala / Python development, emphasising object-oriented principles.

2. Technical Proficiency:

Proficiency in Apache Spark or PySpark for large-scale data processing, Experience with Airflow for workflow orchestration in production environments, Familiarity with Docker for containerisation and Kubernetes for container orchestration, Experience managing AWS services such as S3, EMR, Glue, Athena, and Redshift, Strong background in SQL and relational databases.

3. Communication Skills:

Excellent English language communication skills, both verbal and written, Ability to collaborate effectively with technical and non-technical stakeholders.

Nice-to-have skills: 

Experience with streaming platforms such as Kafka for real-time data processing, Knowledge of Terraform for infrastructure as code implementation in AWS environments.

  • Senior Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Park Senior Villas Chandler Tempo inteiro

    About the RoleWe are seeking a highly skilled Senior Dataops Engineer to join our team at Park Senior Villas Chandler. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines. Your expertise in automating data management processes in Azure Data Factory, AWS Glue or...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal WE ARE META Tempo inteiro

    Unlock Your Potential as a Senior Data Engineer at WE ARE METAWe are seeking a highly skilled Senior Data Engineer to join our team at WE ARE META. As a Senior Data Engineer, you will play a key role in designing, building, and maintaining large-scale data systems that drive business growth and innovation.About the RoleAs a Senior Data Engineer, you will be...

  • Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Data Privacy Tempo inteiro

    Join Our Team as a Data EngineerWe are seeking a highly skilled Data Engineer to join our team at KCS IT. As a Data Engineer, you will play a crucial role in designing, building, and maintaining our data infrastructure.About the RoleThis is a full-time, remote position in Portugal, and we are looking for a candidate with a strong background in Azure cloud...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Agile Engine. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing data warehouse solutions on Snowflake, and developing and maintaining ETL...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Capgemini Tempo inteiro

    Senior Python DeveloperAt Capgemini Engineering, we're seeking a highly skilled Senior Python Developer to join our team of experts in digital and software technology. As a key member of our engineering services, you'll play a crucial role in helping our clients unleash their potential.Your RoleAs a Senior Engineer, you'll be responsible for designing and...

  • Senior Data Engineer

    Há 7 dias

    Porto, Porto, Portugal Devexperts Llc Tempo inteiro

    Senior Data EngineerAt Devexperts LLC, we're seeking a highly skilled Senior Data Engineer to join our Research & Development Team. Our current project involves working with a top US retail broker, focusing on trading experience, finance reports, and risk management.Key Responsibilities:Design, develop, and maintain robust data pipelines using Java within...

  • Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Data Privacy Tempo inteiro

    We're looking for a talented Data Engineer to join our team. As a Data Engineer, you will play a key role in designing, building, and maintaining our data infrastructure. You will work closely with our data scientists and analysts to ensure that our data is accurate, reliable, and secure.Key ResponsibilitiesDesign and implement data pipelines using Azure...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal diconium data Tempo inteiro

    JobbeschreibungWir suchen einen erfahrenen Data Engineer, der unsere Dateninfrastruktur weiterentwickeln und verbessern kann. Als Teil unseres Data Engineering Teams werden Sie an der Entwicklung und Implementierung von Datenpipelines, Datenbanken und Datenarchitekturen arbeiten.VerantwortlichkeitenEntwicklung und Implementierung von Datenpipelines und...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Five9 Tempo inteiro

    Job Title: Senior Data EngineerFive9 is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support our business operations.Responsibilities:Lead the development of data pipelines and architectures to support business...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Agile Engine. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing datawarehouse solutions on Snowflake, and developing and maintaining ETL...

  • Senior Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Devexperts Llc Tempo inteiro

    Job Title: Senior Data EngineerAbout the Role:Devexperts Llc is seeking a highly skilled Senior Data Engineer to join our Research & Development Team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining robust data pipelines using Java within AWS infrastructure. You will also implement scalable solutions for data...

  • Senior Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Tn Portugal. As a Senior Data Engineer, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing data warehouse solutions on Snowflake, and developing and maintaining ETL...

  • Senior Data Engineer

    Há 3 dias

    Porto, Porto, Portugal Aitopics Tempo inteiro

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Planet. As a Senior Data Engineer, you will play a key role in building and maintaining our data infrastructure, with a focus on Snowflake, Fivetran, and dbt.Key Responsibilities:Design and develop efficient and scalable data pipelines using Fivetran for data...

  • Senior Data Infrastructure Specialist

    2 semanas atrás

    Porto, Porto, Portugal Data Privacy Tempo inteiro

    We are seeking a highly skilled Senior Data Infrastructure Specialist to join our team at Data Privacy. The ideal candidate will have a strong background in Big Data Platform Engineering and experience with Azure infrastructure. Key responsibilities include designing and implementing scalable data architectures, ensuring data security and compliance, and...

  • Senior Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Tn Portugal Tempo inteiro

    Job Title: Senior Data EngineerJob Description:At Tn Portugal, we're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling. You will also manage and optimize data warehouse solutions on...

  • Senior Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Kcs It Tempo inteiro

    We're seeking a talented Data Engineer to join our team at KCS IT. As a senior member of our team, you'll be responsible for designing, developing, and maintaining large-scale data systems using PySpark, SQL, and Airflow. Our ideal candidate has a strong background in data engineering, with expertise in PySpark and SQL. Experience with Airflow and GCP is a...

  • Data Engineer

    Há 1 mês

    Porto, Portugal Data Privacy Tempo inteiro

    .We're looking for the special, unique, and amazing YOU! @ KCS IT, we look for the ones that stands out, for those that always wants to be better and fight for it, and for those who has the same values that we do: dedication, energy, integrity, transparency, flexibility, trust, honesty, hard work, proactivity, teamwork. At KCS we stand for equality and value...

  • Data Engineer

    2 semanas atrás

    Porto, Porto, Portugal Data Privacy Tempo inteiro

    We're seeking a talented Data Engineer to join our team at KCS IT. As a key member of our team, you will be responsible for designing, implementing, and maintaining large-scale data processing systems using Azure cloud technologies. Your expertise in Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics will be crucial in...

  • Data Engineer

    3 semanas atrás

    Porto, Porto, Portugal Data Privacy Tempo inteiro

    About the RoleWe're seeking a talented Data Engineer to join our team at KCS IT. As a Data Engineer, you will play a crucial role in designing, building, and maintaining large-scale data systems.Key ResponsibilitiesDesign and implement data pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.Develop and maintain data models, ETL...

  • Senior Data Engineer Officer

    1 dia atrás

    Porto, Porto, Portugal Match Profiler Tempo inteiro

    Job Title: Senior Data Engineer OfficerAbout Us: Match Profiler is a consulting firm with multidisciplinary competences in IT, empowering clients through optimized solutions.Job Summary: We are seeking a Senior Data Engineer Officer to join our team. This role requires a deep understanding of software development cycles, data manipulation, and complex...