Senior Python Engineer, DataHub Ingestion Framework (Palo Alto) Job at Acryl Data, Palo Alto, CA

VmJ6dUl0bDl3L1c3OGh1eEUzanlGd0I3bVE9PQ==
  • Acryl Data
  • Palo Alto, CA

Job Description

DataHub, built by Acryl Data, is an AI & Data Context Platform adopted by over 3,000 enterprises, including Apple, CVS Health, Netflix, and Visa. Innovated jointly with a thriving open-source community of 13,000+ members, DataHub's metadata graph provides in-depth context of AI and data assets with best-in-class scalability and extensibility.

The company's enterprise SaaS offering, DataHub Cloud, delivers a fully managed solution with AI-powered discovery, observability, and governance capabilities. Organizations rely on DataHub solutions to accelerate time-to-value from their data investments, ensure AI system reliability, and implement unified governance, enabling AI & data to work together and bring order to data chaos.

The Challenge

As AI and data products become business-critical, enterprises face a metadata crisis:

  • No unified way to track the complex data supply chain feeding AI systems
  • Engineering teams struggling with data discovery, lineage, and governance
  • Organizations needing machine-scale metadata management, not just human-browsable catalogs

Why This Matters

This is where infrastructure meets impact. The metadata layer you'll build will directly power the next generation of AI systems at massive scale. Your code will determine how safely and effectively thousands of organizations deploy AI, affecting millions of users worldwide.

The Role

We're looking for an exceptional Python engineer to lead development of DataHub's ingestion framework – the core that connects diverse data systems and powers our metadata collection capabilities.

You'll Build

  • Clean, intuitive APIs for our connector ecosystem
  • Event-driven architectures for real-time metadata processing
  • Schema mapping between diverse systems and DataHub's unified model
  • Versioning systems for AI assets (training data, model weights, embeddings)

You Have

  • 4+ years building production-grade distributed systems
  • Advanced Python expertise with a focus on API design
  • Experience with high-scale data processing or integration frameworks
  • Strong systems knowledge and distributed architecture experience
  • A track record of solving complex technical challenges

Bonus Points

  • Experience with DataHub or similar metadata/ETL frameworks (Airflow, Airbyte, dbt)
  • Open-source contributions
  • Early-stage startup experience

Location and Compensation

Bay Area (hybrid, 3 days in Palo Alto office)

Salary Range: $225,000 to $300,000

  • Equity
  • Medical, dental, and vision insurance (99% coverage for employees, 65% coverage for dependents; USA-based employees)
  • Carrot Fertility Program (USA-based employees)
  • Remote friendly
  • Work from home and monthly co-working space budget

Apply for this job

*

indicates a required field

First Name *

Last Name *

Preferred First Name

Email *

Phone *

Resume/CV *

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

#J-18808-Ljbffr

Job Tags

Work at office, Remote work, Work from home, Worldwide,

Similar Jobs

Comphealth

Locum | Physician General Surgery Job at Comphealth

Some locum assignments can be as short as a day, others, years. Some are far from home, others are local. Whatever it is you're looking for, we offer true opportunities, not just postings. CompHealth goes far beyond a job board, providing you with expert guidance tailored...

Emplify Health by Gundersen

Laboratory Support Tech/Phlebotomist | Every Other Weekend - Days Job at Emplify Health by Gundersen

 ...core laboratory team as a casual on-call day phlebotomist at our main hospital campus...  ...not required as we will provide on-the-job training! The Laboratory Support Technician/...  ...tasks and work independently, and previous phlebotomy/medical experience preferred but not required... 

Avera

Physical Therapy Tech | PRN Job at Avera

 ...Under the supervision of a therapist, provides support to therapy services. Duties may include preparing the patient and the treatment area, clerical duties such as registration, billing, scheduling, computer and answering the phone, transporting patients, cleaning and... 

Mitch Hollier - State Farm Agent

Assistant Office Manager - State Farm Agent Team Member Job at Mitch Hollier - State Farm Agent

 ...questions, this may be the career for you Responsibilities Find prospects Quote prospects close prospects As an Agent Team Member, you will receive... ~401(k)~ Group Life Insurance Benefits ~ Salary plus commission/bonus ~ Health benefits ~ Paid... 

Tri-State Horizons Inc.

Entry Level Marketing Manager Trainee Job at Tri-State Horizons Inc.

 ...Job Description Job Description Entry Level Sales/ Marketing Manager - Weekly pay $600 - $1200 West Columbia, SC Immediate Start No Experience Required Degree Not Required If you are bored of having a job and want a career, then this is the opportunity...