Dataops Engineer

We are looking for a skilled DataOps Engineer to join our team. As a DataOps Engineer, you will be responsible for ensuring that our data infrastructure is running smoothly and efficiently. You will be responsible for responding to incidents, maintaining our infrastructure, optimizing databases for cost and speed, managing our infrastructure, and deploying new data flows.
Requested Skills:
• Google Cloud Platform: General knowledge of the platform and various services, and at least one year of experience with GCP.
• Apache Airflow: At least two years of experience with the Airflow orchestrator, experience with Google Composer is a plus.
• Google BigQuery: Extensive experience (at least 4 years) with GBQ, know how to optimize tables and queries, and able to design database architecture.
• Terraform: At least two years of experience with Terraform, and know good practices of GitOps.
• Apache Spark: this is an optional expertise we would value. Some of our pipelines use pySpark.
• Additional Knowledge and Experience that are a Plus: o Pub/Sub o Kafka o Azure Analysis Services o Google Cloud Storage optimization.
Main Responsibilities:
Incident Response o Understand problems from a user perspective and communicate to clearly understand the issue.
Reproduce bugs or issues that users are facing.
Apply root cause analysis to quickly and efficiently
Find the root cause of the problem, patch it, test it, and communicate with the end user.
Write postmortems summarizing every step of resolution and helping the team to track all issues.
Monitor existing flows and infrastructure and perform the same tasks when discovering bugs/issues through monitoring and alerting.
Maintenance o Monitor flows and infrastructure to identify potential issues.
Adapt configurations to keep flows and infrastructures working as expected, keeping the operations without incident.
o Track costs and time of processing through dedicated dashboards.
o Alert people who query tables the wrong way, involving high costs.
o Track down jobs, views, and tables that are running inefficiently and occur either high costs or low speed of execution.
o Optimize jobs, queries, and tables to optimize both costs and speed of execution.
• Infrastructure Management o Manage infrastructure through Terraform.
o Share and propose good practices.
o Decommission useless infrastructures such as services, tables, or virtual machines.
o Track future deployments with a Data Architect and participate in Deployment Reviews.
o Share and propose good practices of deployment.
o Accompany Data Engineers during the entire process of deployments. o Accompany Data Engineers in the following period of active monitoring.
o Ensure diligent application of deployment process, logging, and monitoring strategy.
o Take over newly deployed flows in the run process.
First Name
Last Name
Email Address
Phone Number +48 Poland (Polska) +48
Afghanistan (?????????) +93
Albania (Shqipëri) +355
Algeria (???????) +213
American Samoa +1
Andorra +376
Angola +244
Anguilla +1
Antigua and Barbuda +1
Argentina +54
Armenia (????????) +374
Aruba +297
Australia +61
Austria (Österreich) +43
Azerbaijan (Az?rbaycan) +994
Bahamas +1
Bahrain (???????) +973
Bangladesh (????????) +880
Barbados +1
Belarus (????????) +375
Belgium (België) +32
Belize +501
Benin (Bénin) +229
Bermuda +1
Bhutan (?????) +975
Bolivia +591
Bosnia and Herzegovina (????? ? ???????????) +387
Botswana +267
Brazil (Brasil) +55
British Indian Ocean Territory +246
British Virgin Islands +1
Brunei +673
Bulgaria (????????) +359
Burkina Faso +226
Burundi (Uburundi) +257
Cambodia (???????) +855
Cameroon (Cameroun) +237
Canada +1
Cape Verde (Kabu Verdi) +238
Caribbean Netherlands +599
Cayman Islands +1
Central African Republic (République centrafricaine) +236
Chad (Tchad) +235
Chile +56
China (??) +86
Christmas Island +61
Cocos (Keeling) Islands +61
Colombia +57
Comoros (??? ?????) +269
Congo (DRC) (Jamhuri ya Kidemokrasia ya Kongo) +243
Congo (Republic) (Congo-Brazzaville) +242
Cook Islands +682
Costa Rica +506
Côte d'Ivoire +225
Croatia (Hrvatska) +385
Cuba +53
Curaçao +599
Cyprus (??p???) +357
Czech Republic (Ceská republika) +420
Denmark (Danmark) +45
Djibouti +253
Dominica +1
Dominican Republic (República Dominicana) +1
Ecuador +593
Egypt (???) +20
El Salvador +503
Equatorial Guinea (Guinea Ecuatorial) +240
Eritrea +291
Estonia (Eesti) +372
Ethiopia +251
Falkland Islands (Islas Malvinas) +500
Faroe Islands (Føroyar) +298
Fiji +679
Finland (Suomi) +358
France +33
French Guiana (Guyane française) +594
French Polynesia (Polynésie française) +689
Gabon +241
Gambia +220
Georgia (??????????) +995
Germany (Deutschland) +49
Ghana (Gaana) +233
Gibraltar +350
Greece (????da) +30
Greenland (Kalaallit Nunaat) +299
Grenada +1
Guadeloupe +590
Guam +1
Guatemala +502
Guernsey +44
Guinea (Guinée) +224
Guinea-Bissau (Guiné Bissau) +245
Guyana +592
Haiti +509
Honduras +504
Hong Kong (??) +852
Hungary (Magyarország) +36
Iceland (Ísland) +354
India (????) +91
Indonesia +62
Iran (?????) +98
Iraq (??????) +964
Ireland +353
Isle of Man +44
Israel (?????) +972
Italy (Italia) +39
Jamaica +1
Japan (??) +81
Jersey +44
Jordan (??????) +962
Kazakhstan (?????????) +7
Kenya +254
Kiribati +686
Kosovo +383
Kuwait (??????) +965
Kyrgyzstan (??????????) +996
Laos (???) +856
Latvia (Latvija) +371
Lebanon (?????) +961
Lesotho +266
Liberia +231
Libya (?????) +218
Liechtenstein +423
Lithuania (Lietuva) +370
Luxembourg +352
Macau (??) +853
Macedonia (FYROM) (??????????) +389
Madagascar (Madagasikara) +261
Malawi +265
Malaysia +60
Maldives +960
Mali +223
Malta +356
Marshall Islands +692
Martinique +596
Mauritania (?????????) +222
Mauritius (Moris) +230
Mayotte +262
Mexico (México) +52
Micronesia +691
Moldova (Republica Moldova) +373
Monaco +377
Mongolia (??????) +976
Montenegro (Crna Gora) +382
Montserrat +1
Morocco (??????) +212
Mozambique (Moçambique) +258
Myanmar (Burma) (??????) +95
Namibia (Namibië) +264
Nauru +674
Nepal (?????) +977
Netherlands (Nederland) +31
New Caledonia (Nouvelle-Calédonie) +687
New Zealand +64
Nicaragua +505
Niger (Nijar) +227
Nigeria +234
Niue +683
Norfolk Island +672
North Korea (?? ???? ?? ???) +850
Northern Mariana Islands +1
Norway (Norge) +47
Oman (?????) +968
Pakistan (???????) +92
Palau +680
Palestine (??????) +970
Panama (Panamá) +507
Papua New Guinea +675
Paraguay +595
Peru (Perú) +51
Philippines +63
Poland (Polska) +48
Portugal +351
Puerto Rico +1
Qatar (???) +974
Réunion (La Réunion) +262
Romania (România) +40
Russia (??????) +7
Rwanda +250
Saint Barthélemy +590
Saint Helena +290
Saint Kitts and Nevis +1
Saint Lucia +1
Saint Martin (Saint-Martin (partie française)) +590
Saint Pierre and Miquelon (Saint-Pierre-et-Miquelon) +508
Saint Vincent and the Grenadines +1
Samoa +685
San Marino +378
São Tomé and Príncipe (São Tomé e Príncipe) +239
Saudi Arabia (??????? ??????? ????????) +966
Senegal (Sénégal) +221
Serbia (??????) +381
Seychelles +248
Sierra Leone +232
Singapore +65
Sint Maarten +1
Slovakia (Slovensko) +421
Slovenia (Slovenija) +386
Solomon Islands +677
Somalia (Soomaaliya) +252
South Africa +27
South Korea (????) +82
South Sudan (???? ???????) +211
Spain (España) +34
Sri Lanka (???? ?????) +94
Sudan (???????) +249
Suriname +597
Svalbard and Jan Mayen +47
Swaziland +268
Sweden (Sverige) +46
Switzerland (Schweiz) +41
Syria (?????) +963
Taiwan (??) +886
Tajikistan +992
Tanzania +255
Thailand (???) +66
Timor-Leste +670
Togo +228
Tokelau +690
Tonga +676
Trinidad and Tobago +1
Tunisia (????) +216
Turkey (Türkiye) +90
Turkmenistan +993
Turks and Caicos Islands +1
Tuvalu +688
U.S. Virgin Islands +1
Uganda +256
Ukraine (???????) +380
United Arab Emirates (???????? ??????? ???????) +971
United Kingdom +44
United States +1
Uruguay +598
Uzbekistan (O?zbekiston) +998
Vanuatu +678
Vatican City (Città del Vaticano) +39
Venezuela +58
Vietnam (Vi?t Nam) +84
Wallis and Futuna (Wallis-et-Futuna) +681
Western Sahara (??????? ???????) +212
Yemen (?????) +967
Zambia +260
Zimbabwe +263
Åland Islands +358

#J-18808-Ljbffr