Dataops Engineer

1 semana atrás


Lisboa, Lisboa, Portugal Emagine Tempo inteiro
We are looking for a skilled DataOps Engineer to join our team. As a DataOps Engineer, you will be responsible for ensuring that our data infrastructure is running smoothly and efficiently.

You will be responsible for responding to incidents, maintaining our infrastructure, optimizing databases for cost and speed, managing our infrastructure, and deploying new data flows.

Remote work + 1 day onboarding in Leiria + preferably occasional visits in the office later on (approx.

once a month)Requested Skills:

Google Cloud Platform:
General knowledge of the platform and various services, and at least one year of experience with GCP.

Apache Airflow:
At least two years of experience with the Airflow orchestrator, experience with Google Composer is a plus.

Google BigQuery:

Extensive experience (at least 4 years) with GBQ, know how to optimize tables and queries, and able to design database architecture.


Terraform:
At least two years of experience with Terraform, and know good practices of GitOps.

Apache Spark:
this is an optional expertise we would value. Some of our pipelines use pySpark.

  • Additional Knowledge and Experience that are a Plus: o Pub/Sub o Kafka o Azure Analysis Services o Google Cloud Storage optimization

Main Responsibilities:
Incident Response o Understand problems from a user perspective and communicate to clearly understand the issue.
Reproduce bugs or issues that users are facing.

Apply root cause analysis to quickly and efficientlyFind the root cause of the problem, patch it, test it, and communicate with the end user.

Write postmortems summarizing every step of resolution and helping the team to track all issues.
Monitor existing flows and infrastructure and perform the same tasks when discovering bugs/issues through monitoring and alerting.
Maintenance o Monitor flows and infrastructure to identify potential issues.
Adapt configurations to keep flows and infrastructures working as expected, keeping the operations without incident.

  • Database Optimizationo Track costs and time of processing through dedicated dashboards.o Alert people who query tables the wrong way, involving high costs.o Track down jobs, views, and tables that are running inefficiently and occur either high costs or low speed of execution.o Optimize jobs, queries, and tables to optimize both costs and speed of execution.
  • Infrastructure Management o Manage infrastructure through Terraform.o Share and propose good practices.o Decommission useless infrastructures such as services, tables, or virtual machines.
  • Deploymentso Track future deployments with a Data Architect and participate in Deployment Reviews.o Share and propose good practices of deployment.o Accompany Data Engineers during the entire process of deployments. o Accompany Data Engineers in the following period of active monitoring.o Ensure diligent application of deployment process, logging, and monitoring strategy.o Take over newly deployed flows in the run process.#J-18808-Ljbffr

  • DataOps Engineer

    Há 2 dias


    Lisboa, Lisboa, Portugal Match Profiler Tempo inteiro

    DataOps Engineer (m/f) – Remote Descrição da função Match Profiler is an Information Systems consultant who has been working in the national and international market since 1999. We have multidisciplinary skills and IT know-how that allow us to contribute, optimize and progress with our clients. We are currently recruiting, for integration in our...

  • Senior Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal BJSS Tempo inteiro

    About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you'll help our clients deploy data...

  • Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal Efficio Tempo inteiro

    Description Please note this role is a non-consulting position. In this job you will develop and implement data pipelines on AWS, maintain and develop them further under guidance of business owners. You will work alongside Data Scientists, Data Engineers and others; integrated in our Product and Digital team. This role offers the chance to join Efficio...

  • Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal Psicotec, Sa Tempo inteiro

    We are looking for an extraordinary professional to be Data Team Leader in a company that creates solutions in the area of Analytics, Business Intelligence & Big Data. This company works for IBEX-35 organizations. In this organization you have the opportunity to set your time, talk about new challenges every 6 months and a permanent contract, because they...

  • Senior Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal BJSS Tempo inteiro

    About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value.As a BJSS data engineer you'll help our clients deploy data...

  • Senior Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal Bjss Tempo inteiro

    We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards, including our most...

  • Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal Sdg Group Tempo inteiro

    Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineeringAt least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with on-premise or cloud toolsExperience in data architecture (structure, partitioning, modeling,...

  • Data Engineer

    1 semana atrás


    Lisboa, Lisboa, Portugal SDG Group Tempo inteiro

    Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineering At least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with onpremise or cloud tools Experience in data architecture (structure, partitioning, modeling,...


  • Lisboa, Portugal Grit Solutions Tempo inteiro

    Senior DataOps Engineer Building Data Highways: DataOps Engineer Available! There's no secret for being number-one; it takes willingness to improve and determination to pursue it. At GRiT®,?it's all about the journey to get there. We believe talent and intelligence are important, yet passion and perseverance are the key to achieving any goal. We are Tech...

  • Dataops Engineer

    Há 5 dias


    Lisboa, Portugal Emagine Tempo inteiro

    We are looking for a skilled DataOps Engineer to join our team. As a DataOps Engineer, you will be responsible for ensuring that our data infrastructure is running smoothly and efficiently. You will be responsible for responding to incidents, maintaining our infrastructure, optimizing databases for cost and speed, managing our infrastructure, and deploying...


  • Lisboa, Portugal Hays Tempo inteiro

    Principais responsabilidades Garantir o crescimento global da empresa e tornar a tomada de decisões mais rápida e informada através do fornecimento de dados e análises avançadas; Garantir a disponibilidade e acessibilidade dos dados; Criar e gerir ambientes de dados na cloud, nomeadamente em produção; Planear e fornecer estratégias de integração...


  • Lisboa, Portugal Bjss Tempo inteiro

    .We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won several awards, including our most...

  • Dataops Engineer

    Há 2 dias


    Lisboa, Portugal Genesis Studio Llc. Tempo inteiro

    Hibrido (algumas vezes por meses), Portugal | Publicado em 28-05-2024Estamos a reforçaruma das nossas equipas em modelo híbrido em Lisboa, com um perfil de DataOpsEngineer:Requisitos • Mínimo 4 anos deexperiência em funções similares;• Experiência em GCPe Apache Airflow;• Experiência emGoogle BigQuery, para optimizar tabelas, consultas e...

  • Data Engineer

    Há 5 dias


    Lisboa, Portugal Bjss Tempo inteiro

    About UsWe're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards, including a...

  • Data Engineer

    Há 5 dias


    Lisboa, Portugal Bjss Tempo inteiro

    About UsWe're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards, including a...


  • Lisboa, Portugal Bjss Tempo inteiro

    About UsWe're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day.In the last 30 years we won several awards, including our...


  • Lisboa, Portugal Bjss Tempo inteiro

    .About the Role BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you'll help our clients deploy data...

  • Data Engineer

    Há 4 dias


    Lisboa, Portugal Sdg Group Tempo inteiro

    .Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineering At least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with on-premise or cloud tools Experience in data architecture (structure, partitioning, modeling,...


  • Lisboa, Portugal Bjss Tempo inteiro

    About the RoleBJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software engineering with the knowledge and experience of building data solutions in order to deliver business value.As a BJSS data engineer you'll help our clients deploy data...


  • Lisboa, Portugal Siemens S.A. Tempo inteiro

    Digital value creator.We have lots of ideas about how to successfully drive digitalization in companies.For example, with open cloud platforms, highly developed security systems, and clever tools for developers.What is your role in this?Move the world from behind the scenes with your IT expertise and passion for game-changing information technology.Rethink...


  • Lisboa, Portugal Bjss Tempo inteiro

    We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won several awards, including our most...

  • Senior Data Engineer

    4 semanas atrás


    Lisboa, Portugal Bjss Tempo inteiro

    We're an award-winning innovative tech consultancy - a team of creative problem solvers. Since 1993 we've been finding better, more sustainable ways to solve complex technology problems for some of the world's leading organisations and delivered solutions that millions of people use every day. In the last 30 years we won several awards, including our most...

  • Data Engineer

    Há 1 mês


    Lisboa, Portugal Psicotec, Sa Tempo inteiro

    We are looking for an extraordinary professional to be Data Team Leader in a company that creates solutions in the area of Analytics, Business Intelligence & Big Data. This company works for IBEX-35 organizations. In this organization you have the opportunity to set your time, talk about new challenges every 6 months and a permanent contract, because they...

  • Data Engineer

    4 semanas atrás


    Lisboa, Portugal Psicotec, Sa Tempo inteiro

    We are looking for an extraordinary professional to be Data Team Leader in a company that creates solutions in the area of Analytics, Business Intelligence & Big Data. This company works for IBEX-35 organizations. In this organization you have the opportunity to set your time, talk about new challenges every 6 months and a permanent contract, because they...

  • Data Engineer

    Há 1 mês


    Lisboa, Portugal Sdg Group Tempo inteiro

    Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineeringAt least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with on-premise or cloud toolsExperience in data architecture (structure, partitioning, modeling,...

  • Data Engineer

    Há 5 dias


    Lisboa, Portugal Sdg Group Tempo inteiro

    Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineeringAt least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with on-premise or cloud toolsExperience in data architecture (structure, partitioning, modeling,...

  • Data Engineer

    4 semanas atrás


    Lisboa, Portugal Sdg Group Tempo inteiro

    Degree in computer science or telecommunications engineering, or any other degree in a technical environment with proximity to software engineeringAt least 4 years of experience in a data engineer role implementing data ingestion and transformation solutions with on-premise or cloud toolsExperience in data architecture (structure, partitioning, modeling,...


  • Lisboa, Portugal Black Pen Recruitment Tempo inteiro

    Relocate to Malta GCP Data and ML Engineer (Consulting/Big 4) Lisbon, Lisbon, Portugal Job Openings Relocate to Malta GCP Data and ML Engineer (Consulting/Big 4) About the job Relocate to Malta GCP Data and ML Engineer (Consulting/Big 4)Our clients Microsoft Business Solutions team is a Microsoft Gold Partner and leader in Microsoft software implementations...