Senior Data Engineer (m/f/d)

You build and run the pipelines behind GlassDollar's company data: ingestion, scraping, enrichment, entity resolution, LLM extraction and signal detection – from first prototype to reliable production. You sit in the product team, report to our Product Owner and work closely with our engineers.
Our data has grown faster than our engineering around it. Feeds run, but nothing tells us when they stop. Most relationship records aren't classified yet, duplicates slip through, and some signals users ask for, like headcount changes, have no reliable live source. Making this robust, observable and fast is the job.

Who uses your work

• People in vclOS: Search, lists, projects, PoC tracking, recommendations and weekly company updates.
• Our AI agents & search: Clean fields with provenance, confidence and freshness, measured on our search benchmark.
• API and MCP partners: Stable schemas and fields to build on.
• Marketing: Directory, rankings and reports through a quarterly data release.

When these conflict, the product wins.

What you own

• Ingestion and pipelines: vendor feeds, APIs and scraping jobs – scheduled, idempotent, monitored, with backfills and alerts when a feed stops or drifts.
• Schema and data model: the Postgres schema and data dictionary as the source of truth, including migrations and the Hasura layer on top.
• Entity resolution: matching and deduplicating companies across sources, with measurable precision and recall.
• LLM extraction and classification: pipelines that turn unstructured sources into structured records, evaluated on labelled samples and tuned for cost per record.
• Signals: funding, acquisitions, shutdowns, case studies and headcount as event pipelines with change detection and correct event dates.
• Data quality and observability: automated tests and metrics for coverage, accuracy, freshness and completeness.
• Data sources: you test vendors, APIs and scraping partners on real samples and give a clear recommendation on quality and cost per datapoint.

You join sprint planning and review, triage data errors reported by users and agents, and take datapoints from prototype to production yourself.

Your first 90 days

By day 90: every feed is monitored and alerting, relationship data is classified with measured precision, entity resolution has a baseline, 3–5 new datapoints are live in production, and our search benchmark has moved against a target you set in month 1.

Month 1: baseline and observability

• Inventory every pipeline, vendor feed and scraping job; give each one an owner, runbook, monitoring and alerting.
• Data dictionary v1 for the top 25 datapoints, with source, lineage and quality numbers.
• Automated quality checks (coverage, freshness, nulls, duplicates) running on every load.
• Quick win: Scale partnership data point collection.

Month 2: corporate–technology relationships

• Model relationship types in the schema: mention, partnership, PoC, implementation, investment, acquisition.
• Build an LLM classification pipeline for existing and incoming records; report precision on a labelled sample and cost per record.
• Set a recall and lag baseline, then improve the relationship pipeline against it.
• Integrate a live acquisition source and keep acquired companies out of recommendations.

Month 3: new datapoints, entity resolution and agents

• Run 6–8 time-boxed datapoint experiments, each ending in ship, kill or park. Ship 3–5 to production.
• Entity resolution v2: measurable matching across sources, with a review queue for uncertain cases.
• Add provenance, confidence and freshness to the top 25 fields, and make customer data (lists, PoCs, notes) usable by agents with proper permissions.
• Document the data platform: architecture, runbooks and on-call handover.

Who you are

A senior data engineer who ships. You care about reliability and clean data models, and you understand what the data is used for. You'd rather put a rough datapoint live this week than a perfect one next quarter.

Must-haves

• 5+ years in data engineering (or backend engineering with a data focus), ideally with company, market or B2B data at a data business, marketplace or SaaS company.
• You've built and run production data pipelines end to end: orchestration (e.g. Airflow, Dagster or Prefect), monitoring, alerting and backfills.
• Fluent Python: scrapers and enrichment services (requests or Playwright, pandas or polars), tested and production-ready.
• Strong SQL on a production Postgres database: schema design, migrations, indexing and query performance.
• Hands-on with LLMs for extraction and classification, measured on a labelled sample, with an eye on cost and latency.
• Practical entity resolution and deduplication at scale.
• Data quality testing (e.g. dbt tests, Great Expectations or your own checks).
• Working knowledge of GDPR for company and people data.
• Based in Berlin, or willing to relocate.

Nice to have

• dbt, Docker and a major cloud (AWS or GCP); infrastructure as code.
• Event or signal pipelines: change detection, event dating, alerting.
• Vector search or embeddings for matching and retrieval.
• GraphQL / Hasura.
• German at reading level: DACH registers are a big part of our upside.
• Working with data from providers such as PitchBook, Crunchbase, Dealroom or company registers.
• Knowledge of the corporate innovation, VC or emerging technology ecosystem.

Stack: Postgres, Hasura, Python, PostHog and LLM APIs.

What this role is not:

• Not BI or analytics: no dashboards for their own sake.
• Not ML research: you use models, you don't train them.
• Not customer workspace data (projects, departments, exports): that stays with product engineering.
• Not a ticket-taker: you decide what's worth building, not just how.
• Not team-building from day one: you lead through scope and technical quality.