Senior Data Engineer
4 semanas atrás
Job Title: Senior Data Engineer
Company Description: Tn Portugal works with respected financial institutions, delivering products and solutions for retail and brokerage houses, exchanges, and firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.
Job Description:
We are looking for a Senior Data Engineer with a Java/Scala/Python background to join our Research & Development Team. Our current project is for a Top-5 US retail broker (by the number of users). The project is devoted to trading experience, finance reports, and risk management.
Responsibilities:
1. Develop Data Pipeline: Design, develop, and maintain robust data pipelines using Java within AWS infrastructure. Implement scalable solutions for data analysis and transformation using Apache Spark and PySpark. Utilise Airflow for efficient workflow orchestration in complex data processing tasks. Ensure fast and interactive querying capabilities through the use of Presto.
2. Manage Infrastructure: Containerise applications using Docker for streamlined deployment and scaling. Orchestrate and manage containers effectively with Kubernetes in production environments. Implement infrastructure as code using Terraform for provisioning and managing AWS resources.
3. Collaborate and Communicate: Collaborate with functional teams to understand data requirements and architect scalable solutions aligned with business goals. Ensure data quality and reliability through robust testing methodologies and monitoring solutions. Stay updated with emerging technologies and industry trends to continuously enhance the data engineering ecosystem.
Qualifications:
Must-have skills:
1. Education and Experience: Bachelor's degree in Computer Science, Engineering, or related field. Minimum 5 years of experience in Java/Scala/Python development, emphasising object-oriented principles.
2. Technical Proficiency: Proficiency in Apache Spark or PySpark for large-scale data processing. Experience with Airflow for workflow orchestration in production environments. Familiarity with Docker for containerisation and Kubernetes for container orchestration. Experience managing AWS services such as S3, EMR, Glue, Athena, and Redshift. Strong background in SQL and relational databases.
3. Communication Skills: Excellent English language communication skills, both verbal and written. Ability to collaborate effectively with technical and non-technical stakeholders.
Nice-to-have skills: Experience with streaming platforms such as Kafka for real-time data processing. Knowledge of Terraform for infrastructure as code implementation in AWS environments.
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Park Senior Villas Chandler Tempo inteiroAbout the RoleWe are seeking a highly skilled Senior Dataops Engineer to join our team at Park Senior Villas Chandler. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines. Your expertise in automating data management processes in Azure Data Factory, AWS Glue or...
-
Senior Data Engineer
3 semanas atrás
Porto, Porto, Portugal Devexperts Llc Tempo inteiroSenior Data EngineerAt Devexperts LLC, we're seeking a highly skilled Senior Data Engineer to join our Research & Development Team. Our current project involves working with a top US retail broker, focusing on trading experience, finance reports, and risk management.Key Responsibilities:Design, develop, and maintain robust data pipelines using Java within...
-
Data Engineer
4 semanas atrás
Porto, Porto, Portugal Data Privacy Tempo inteiroWe're looking for a talented Data Engineer to join our team. As a Data Engineer, you will play a key role in designing, building, and maintaining our data infrastructure. You will work closely with our data scientists and analysts to ensure that our data is accurate, reliable, and secure.Key ResponsibilitiesDesign and implement data pipelines using Azure...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Devexperts Llc Tempo inteiroJob Title: Senior Data EngineerAbout the Role:Devexperts Llc is seeking a highly skilled Senior Data Engineer to join our Research & Development Team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining robust data pipelines using Java within AWS infrastructure. You will also implement scalable solutions for data...
-
Senior Data Engineer
3 semanas atrás
Porto, Porto, Portugal Aitopics Tempo inteiroSenior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our team at Planet. As a Senior Data Engineer, you will play a key role in building and maintaining our data infrastructure, with a focus on Snowflake, Fivetran, and dbt.Key Responsibilities:Design and develop efficient and scalable data pipelines using Fivetran for data...
-
Senior Data Engineer
1 semana atrás
Porto, Porto, Portugal Swordhealth Tempo inteiroJob SummaryWe are seeking a talented Senior Data Engineer to join our team at Swordhealth. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining ETL/ELT data pipelines that ingest data from multiple sources with different technologies.Main Responsibilities• Design, build, and maintain ETL/ELT data pipelines that ingest...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data EngineerJob Description:At Tn Portugal, we're seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling. You will also manage and optimize data warehouse solutions on...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Kcs It Tempo inteiroWe're seeking a talented Data Engineer to join our team at KCS IT. As a senior member of our team, you'll be responsible for designing, developing, and maintaining large-scale data systems using PySpark, SQL, and Airflow. Our ideal candidate has a strong background in data engineering, with expertise in PySpark and SQL. Experience with Airflow and GCP is a...
-
Porto, Porto, Portugal Data Privacy Tempo inteiroWe are seeking a highly skilled Senior Data Infrastructure Specialist to join our team at Data Privacy. The ideal candidate will have a strong background in Big Data Platform Engineering and experience with Azure infrastructure. Key responsibilities include designing and implementing scalable data architectures, ensuring data security and compliance, and...
-
Data Engineer
Há 1 mês
Porto, Porto, Portugal Data Privacy Tempo inteiroWe're seeking a talented Data Engineer to join our team at KCS IT. As a key member of our team, you will be responsible for designing, implementing, and maintaining large-scale data processing systems using Azure cloud technologies. Your expertise in Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics will be crucial in...
-
Senior Data Engineer Officer
2 semanas atrás
Porto, Porto, Portugal Match Profiler Tempo inteiroJob Title: Senior Data Engineer OfficerAbout Us: Match Profiler is a consulting firm with multidisciplinary competences in IT, empowering clients through optimized solutions.Job Summary: We are seeking a Senior Data Engineer Officer to join our team. This role requires a deep understanding of software development cycles, data manipulation, and complex...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data EngineerAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing data pipelines using DBT for transformation and modeling, managing and optimizing data warehouse solutions on Snowflake, and developing and...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data Engineer - Cloud Data ArchitectAbout the Role:We are seeking a highly skilled Senior Data Engineer to join our team in Portugal. As a Senior Data Engineer, you will be responsible for designing, implementing, and operating cloud-based data architectures on Amazon Web Services (AWS).Key Responsibilities:Design and implement cloud-based...
-
Senior Data Engineer
3 semanas atrás
Porto, Porto, Portugal Phiture Tempo inteiroAbout the Role:We are seeking an exceptional Senior Data Engineer to join our team at Phiture. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement complex data pipelines using Azure Data Factory and DatabricksDevelop and...
-
Senior Data Engineer Position
3 semanas atrás
Porto, Porto, Portugal Kcs It Tempo inteiroAbout the RoleWe're seeking a Senior Data Engineer with 5+ years of experience to join our team in Porto, Portugal. As a Senior Data Engineer, you will be responsible for designing and developing end-to-end data pipelines using Alteryx, ETL processes, and Data Mesh concepts. You will also have expertise in programming languages such as Python, Java, and...
-
Senior Data Platform Engineer
1 semana atrás
Porto, Porto, Portugal Kuehne+Nagel Tempo inteiroJob Role: Senior Data Platform EngineerAbout Us: Kuehne+Nagel is a leading global logistics company that is shaping the future of logistics and IT.Your Role: As a Senior Data Platform Engineer, you will be part of our central data platform team, designing and implementing our data mesh approach on a cloud transformation program.Responsibilities:Design and...
-
Data Engineer
Há 1 mês
Porto, Porto, Portugal Data Privacy Tempo inteiroWe are seeking a highly skilled Data Engineer to join our team at KCS IT. As a Data Engineer, you will play a key role in designing, building, and maintaining large-scale data processing systems using Azure cloud technologies. Your expertise in Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics will be instrumental in...
-
Senior Data Analytics Engineer
Há 1 mês
Porto, Porto, Portugal Protegrity Tempo inteiroUnlock the Power of Data AnalyticsProtegrity, a leading provider of data protection solutions, is seeking a talented Senior Data Analytics Engineer to join our Business Intelligence Team. This role is a remote opportunity that offers the chance to make a significant impact in a company that prioritizes data-driven decision-making.As a Senior Data Analytics...
-
Senior Data Engineer
4 semanas atrás
Porto, Porto, Portugal Tn Portugal Tempo inteiroJob Title: Senior Data EngineerCompany Description: Devexperts works with respected financial institutions, delivering products and solutions for retail and brokerage houses, exchanges, and firms. The company focuses on trading platforms and brokerage automation, complex software development projects, market data products, and IT consulting services.Job...
-
Senior Data Engineer
Há 1 mês
Porto, Porto, Portugal Capgemini Tempo inteiroAbout the RoleAs a Senior Engineer at Capgemini, you will be part of a dynamic team that designs and implements distributed data processing solutions for various business cases. Your expertise in Python and experience with modern development practices will be crucial in creating scalable and efficient solutions.Your ResponsibilitiesInvestigate and resolve...