Data Infrastructure Engineer

2 meses atrás

Lisbon, Portugal ComplyAdvantage Tempo inteiro
What you will be doing :

We are looking for a highly skilled Data Infrastructure Engineer to join our team. The Data Infrastructure Engineer will be responsible for the management, monitoring, backup, cost management, performance tuning, and deployment of a variety of highly available and fault-tolerant SQL, NoSQL, Kafka, and Spark-based data layer systems running on AWS and GCP. The ideal candidate will have experience working with various data processing tools and cloud platforms.

As a Data Infrastructure Engineer, you will

  • Design, implement, and maintain data layer orchestration systems to expose and manage a series of self-managed and PaaS data layer technologies
  • Manage and monitor data layer components and services on AWS and GCP, ensuring optimal performance and availability
  • Configure and deploy backup solutions for databases, Kafka, and Spark, ensuring data integrity and facilitating disaster recovery
  • Implement cost management strategies for database and data processing infrastructure, optimising resource utilisation and minimising expenses
  • Perform regular performance tuning for databases, Kafka, and Spark, identifying optimization opportunities and implementing improvements. Troubleshoot and resolve issues related to these systems
  • Create and maintain documentation for Data Infrastructure engineering processes, infrastructure, and applications
  • Work closely with other teams to integrate data sources and ensure data consistency, and design and model databases that support application requirements, and adhere to best practices for normalisation and performance.
  • Monitor and manage the capacity of data infrastructure, planning for future growth and scalability; analyse query performance and implement appropriate indexing strategies
  • Provide technical support and guidance to other team members in relation to Data Infrastructure engineering tasks and technologies
  • Collaborate with stakeholders to define and prioritise Data Infrastructure engineering projects, setting realistic timelines and expectations

Our Tech Stack:

ComplyAdvantage is fully cloud-based, with a modern kubernetes-focused tech stack. All compute workloads run in Kubernetes, with clusters in multiple regions to support the needs of our global client base. Our production services are currently based in AWS and GCP.

We make heavy use of Terraform and Helm to define our infrastructure and services, and lean heavily on GitOps paradigms - production and non-production environments are defined in git and changes to these environments (both cloud infrastructure and Kubernetes applications) are managed via git.

Kafka is used widely across teams, and is seen as a core technology in our tech stack. We are focusing on developments to our Kafka and Yugabyte tooling, as well as the other data systems we rely on, such as PostgreSQL, Elasticsearch and Databricks/Spark. To tie in with our multi-cloud plans, we’ll also be developing inter-cloud failover options to be leveraged by our development teams.

Across the wider Technology team, teams build and release containerised applications to support the wide array of activities that our teams are engaged in - from developing low latency client-facing APIs, to machine learning models and data processing pipelines.

About you:

As an experienced Data Infrastructure Engineer, you will have:

  • Significant experience working with AWS and/or GCP core services
  • Significant experience building, scaling and tuning data systems such as Apache Kafka, Apache Spark, Elasticsearch and PostgreSQL-compatible databases
  • An awareness of cost considerations when designing data systems architectures
  • Some experience working with Kubernetes from a cluster admin perspective
  • Experience codifying infrastructure as code in Terraform and/or Helm charts
  • Experience working with observability tooling (logging, monitoring, tracing)
  • Experience with CI tooling, including test pipelines and asset generation and publishing (eg Docker images, helm charts)
  • Awareness of security considerations
  • Your location based in Lisbon

Nice to have:

  • Experience running databases within Kubernetes
  • Experience of using build-test-run CI/CD techniques to IaC.
  • Experience running distributed SQL databases

Education:

  • BSc/BA degree in computer science, engineering or related discipline OR relevant years of experience in required skills.

What’s in it for you?

  • Equity as we want you to have a part of what we are building
  • Unlimited Time Off Policy- A work-life balance and focus on our well-being are critical to keeping us performing at our best
  • We embrace a hybrid approach that requires employees to be in the office for two days a week. We strongly believe that this approach fosters collaboration and enables the building of meaningful relationships
  • You will also get a new starter budget of €235 to kit out your home office
  • Opportunity to work on innovative projects with smart-minded people keen to share their knowledge and continuously improve
  • Annual learning budget of €2200 (prorated based on start date) to drive your performance and career development

About us:

ComplyAdvantage is the financial industry’s leading source of AI-driven financial crime risk data and detection technology. Our mission is to neutralise the risk of money laundering, terrorist financing, corruption, and other financial crime.

More than 1000 companies rely on us to understand the risk of who they’re doing business with through the world’s only global, real-time database of people and companies. Our solutions identify thousands of risk events daily from millions of structured and unstructured data points.

We have five global hubs in New York, London, Singapore, Lisbon and Cluj-Napoca and are backed by Goldman Sachs, Ontario Teachers, Index Ventures, and Balderton Capital.

Since 2014, we have raised over $100 million in funding, and in 2022 alone grew by over 40% to over 500 people globally. Over the next 12 months, as our revenue increases, we plan to increase to 600.

At ComplyAdvantage diversity fuels our rocket ship and our commitment to inclusion across race, gender, age, religion, identity and experience drives us forward every day. We encourage everyone to apply and aspire to consider every application fairly.

We will handle your information in accordance with our Privacy Policy. For further information, please click

  • Data Engineer Senior

    2 semanas atrás

    Lisbon, Portugal Grupo Data Tempo inteiro

    Position: Data Engineer Senior  you will be responsible for designing, developing, and maintaining our data infrastructure. You will work closely with our team of data scientists and analysts to ensure that our data is accurate, accessible, and secure. You will also be responsible for implementing data pipelines and ETL processes to support our data-driven...

  • Data DevOps Engineer

    2 semanas atrás

    Lisbon, Portugal ECF Data, LLC Tempo inteiro

    Randstad Professionals is specialized in the recruitment of middle and senior positions.Our client is a reference company in the market for car travel. Aggregating more than 2,000 owned vehicles plus the motorhomes, motorhomes and other recreational vehicles of numerous individual and professional hosts, they operate in the US and 15 European countries -...

  • Data Center Infrastructure Expert

    Há 3 dias

    Lisbon, Portugal Vodafone Tempo inteiro

    What you’ll do As a Data Center Infrastructure Expert, you will be responsible for driving infrastructure projects as part of Vodafone Data Centers. You will work as part of the Core Implementation team and have responsibility on driving several activities across departments. You will actively participate in the design and development of...

  • Principal Engineer

    Há 1 mês

    Lisbon, Portugal Landing.jobs Tempo inteiro

    At Arcesium (Permanent), in Lisbon, PortugalExpires at: 2024-09-02Remote policy: Partial remoteArcesium launched with the mission to power the entire investment lifecycle. Building on a platform developed and tested by the D. E. Shaw group, we received additional equity backing from a second seed client, Blackstone Alternative Asset Management. J.P. Morgan,...

  • Principal Engineer

    Há 5 dias

    Lisbon, Portugal Landing.jobs Tempo inteiro

    At Arcesium (Permanent), in Lisbon, Portugal Expires at: 2024-10-02 Remote policy: Partial remote Arcesium launched with the mission to power the entire investment lifecycle. Building on a platform developed and tested by the D. E. Shaw group, we received additional equity backing from a second seed client, Blackstone Alternative Asset Management. J.P....

  • Senior Systems Infrastructure Engineer

    Há 1 mês

    Lisbon, Portugal Qlik Tempo inteiro

    Description What makes us Qlik? A Gartner Magic Quadrant Leader for 13 years in a row, Qlik transforms complex data landscapes into actionable insights, driving strategic business outcomes. Serving over 40,000 global customers, our portfolio leverages pervasive data quality and advanced AI/ML capabilities that lead to better decisions, faster. We...

  • DevOps Data Engineer

    4 meses atrás

    Lisbon, Portugal Indie Campers Tempo inteiro

    THE ROLE To guarantee the best-in-class performance of all the algorithms and systems that power Indie’s operations, we must maintain and evolve our platform with strong foundations in security, flexibility, data and A.I.. As a DevOps Data Engineer you are a dedicated and proficient DevOps Engineer with a knack for constructing robust data...

  • Lead Engineer

    Há 5 dias

    Lisbon, Portugal Landing.jobs Tempo inteiro

    At Arcesium (Permanent), in Lisbon, Portugal Expires at: 2024-10-02 Remote policy: Partial remote Arcesium launched with the mission to power the entire investment lifecycle. Building on a platform developed and tested by the D. E. Shaw group, we received additional equity backing from a second seed client, Blackstone Alternative Asset Management. J.P....

  • Data Scientist

    Há 1 mês

    Lisbon, Portugal Open Data Science Tempo inteiro

    Data Scientist / Machine Learning EngineerWe hiring a data scientist to join our team which is driving innovation in hiring. In this role, you will need to combine your software engineering skills with a strong understanding of data-driven problem-solving. You will be responsible for maintaining, evaluating and improving our current models while actively...

  • Lead Engineer

    Há 1 mês

    Lisbon, Portugal Landing.jobs Tempo inteiro

    At Arcesium (Permanent), in Lisbon, PortugalExpires at: 2024-09-02Remote policy: Partial remoteArcesium launched with the mission to power the entire investment lifecycle. Building on a platform developed and tested by the D. E. Shaw group, we received additional equity backing from a second seed client, Blackstone Alternative Asset Management. J.P. Morgan,...

  • Software & Data Engineer

    3 semanas atrás

    Lisbon, Lisbon, Portugal LOAD Tempo inteiro

    We are seeking a professional passionate about innovation and excellence. We value transparency, team spirit, and commitment. If you enjoy working in a dynamic and cheerful environment, this could be the perfect opportunity for you!!The role aims to:  Optimize data storage solutions for easy access and retrieval Ensure data accuracy and reliability through...

  • DevOps Data Engineer

    4 meses atrás

    Lisbon, Portugal Indie campers Tempo inteiro

    ABOUT USIndie Campers, the go-to campervan Marketplace, is on a mission of making road trips available to everyone. Thanks to its digital approach and customer-centric orientation, Indie Campers has developed a strong booking experience and high-quality road trips at affordable prices. With over one million nights rented through our travel platform, we have...

  • IT Infrastructure and Security Engineer

    Há 1 mês

    Lisbon, Portugal 1GLOBAL Tempo inteiro

    1GLOBAL is a global specialty telecommunications company connecting people and devices instantly, anywhere with our unique connectivity solutions. We manage mobile devices and connectivity globally for the world's largest investment banks, enterprises, big tech companies, and IoT businesses.1GLOBAL isn't just another mobile company-it's a powerhouse in the...

  • Network Infrastructure Engineer

    Há 3 dias

    Lisbon, Portugal Mind Source Tempo inteiro

    Local: Remote (Remoto) Descritivo da função: A Mind Source está a reforçar a sua Equipa de Infraestruturas através de um Network Infrastructure Engineer. Perfil: Licenciatura em Engenharia Informática, Informática de Gestão ou similar; Conhecimento e experiência em tecnologias de Data Center (Cisco Nexus e Cisco ACI); ...

  • Global Shares

    3 meses atrás

    Lisbon, Portugal JPMorgan Chase & Co. Tempo inteiro

    As a Cloud Infrastructure Engineer within the Global Shares team at . Morgan Chase, you will be responsible for creating and managing an enterprise-level portfolio of Azure Infrastructure and services. Being a valued member of our team, you will have the opportunity to develop and expand your skills in a dynamic, fast-paced environment, making a significant...

  • Data Scientist

    1 semana atrás

    Lisbon, Portugal Open Data Science Tempo inteiro

    Data Scientist / Machine Learning EngineerWe are hiring a data scientist to join our team which is driving innovation in hiring. In this role, you will need to combine your software engineering skills with a strong understanding of data-driven problem-solving. You will be responsible for maintaining, evaluating, and improving our current models while...

  • Cloud Infrastructure Tech Lead

    Há 4 dias

    Lisbon, Portugal Vodafone Tempo inteiro

    What you’ll do The Cloud Infrastructure Tech Lead role helps teams (such as product teams, platform teams and application teams) to successfully adopt cloud infrastructure and platform services. It is heavily involved in design and implementation activities that result in new or improved cloud-related capabilities, and it brings skills and...

  • Software Engineer

    Há 1 mês

    Lisbon, Portugal Pyth Data Association Tempo inteiro

    We're hiring software engineers to help build the market data infrastructure for future decentralized finance applications. Your responsibilities will include developing on-chain solutions, creating developer SDKs, enhancing tools, and monitoring. You will also be expected to use your knowledge of the DeFi ecosystem to identify and leverage new...

  • Software & Data Engineer

    3 semanas atrás

    Lisbon, Portugal LOAD Tempo inteiro

    We are seeking a professional passionate about innovation and excellence. We value transparency, team spirit, and commitment. If you enjoy working in a dynamic and cheerful environment, this could be the perfect opportunity for you!!The role aims to:  Optimize data storage solutions for easy access and retrieval Ensure data accuracy and reliability through...

  • Data Engineer GCP

    4 meses atrás

    Lisbon, Portugal Richemont Tempo inteiro

    Data Engineer GCP   HOW WILL YOU MAKE AN IMPACT?     In collaboration with our data platform team and management, you will build data pipelines on our cloud platform and will contribute to bring our data projects to life.  In close collaboration with the various domain leads and data product owner, you will qualify needs and...