Backend Engineer II - Databases
About the Company
Grafana Labs is the company behind Grafana Cloud, a fully managed observability platform used by more than 10,000 organizations to improve reliability, resolve incidents faster, and manage telemetry at scale. Built on open source and open standards, Grafana Cloud connects data across different systems and supports observability for modern applications and AI-driven workflows. Grafana Labs operates as a 100% remote company, with team members across more than 40 countries.
About the Role
Grafana Labs is looking for a Senior Engineer to work on Tempo, the open-source distributed tracing backend that powers Grafana Cloud Traces and Grafana Enterprise Traces. Tempo helps teams search traces, generate metrics from spans, and connect tracing data with logs, metrics, and profiles across the Grafana ecosystem. The focus is shifting toward improving product reliability, operational efficiency, query performance, and the APIs that support Grafana's next generation of observability products. You'll take ownership of complex technical projects, help shape Tempo's architecture, and work with engineers and product teams to make the platform more reliable and easier to use at scale. This is a remote position open to candidates in the United States and Canada.
Key Responsibilities
Own significant technical projects from initial problem definition through implementation and rollout, including trace aggregation APIs, autoscaling, customer limits, and query engine improvements. Contribute to the ingestion, storage, query, and metrics-generation components of Tempo, making informed decisions about performance, cost, and complexity. Write and present design documents, participate in design reviews, and collaborate with Staff engineers and technical leads on broader engineering direction. Develop structured, deterministic, and discoverable APIs that support Grafana products, AI-driven assistants, and external integrations. Improve production reliability by working toward service-level objectives (SLOs), reducing operational overhead, automating deployments, and developing actionable alerts. Work on autoscaling, parameterized rollouts, and multi-cell operations as Tempo expands to support larger workloads. Improve query latency, ingestion performance, and the ability to handle longer query ranges, including 30-day queries. Collaborate with Product Managers and teams working on App Observability, Asserts, Drilldown, and Grafana Assistant to understand their needs and deliver the capabilities they require. Support other engineers through code reviews, design feedback, and pairing, particularly with junior and mid-level team members. Participate in on-call rotations for the services you help build, respond to incidents, and contribute to post-incident reviews. Contribute to Tempo as an open-source project by reviewing external contributions and engaging with the community.
Required Qualifications
Demonstrated experience delivering complex, multi-month technical projects, from design and implementation through production operations. Significant hands-on experience building and operating distributed data systems in production, such as ingestion pipelines, storage engines, or query execution systems. Strong software engineering skills, with the ability to write maintainable, reliable, and performant code. Solid experience with Go, or experience in systems languages such as Rust, C, or C++ that provides a foundation for working with Go. Experience owning production services, participating in on-call rotations, reducing operational toil, and working with service-level objectives. A pragmatic approach to solving technical problems, with an ability to break complex work into manageable steps, deliver initial solutions, and improve them through feedback. Clear written and verbal communication skills, including experience contributing to design documents, code reviews, and collaborative engineering work in a remote, asynchronous setting. Preferred Qualifications / Requirements Experience with distributed tracing, OpenTelemetry, or large-scale observability systems. Experience designing query languages, SQL- or TraceQL-like engines, or APIs intended for programmatic use by services and AI agents. Familiarity with columnar storage formats such as Parquet or purpose-built on-disk formats for analytical workloads. Experience operating multi-tenant, multi-cell SaaS infrastructure at scale on Kubernetes. Experience developing systems for AI and large language model consumers, including structured APIs, metadata discovery, deterministic outputs, and evaluation frameworks. Open-source contributions or project maintainership, with an interest in engaging with developer communities. Experience using Grafana, Prometheus, Loki, or Tempo in an on-call role or through a home lab. Experience working with fully remote, globally distributed teams. Salary & Benefits The US on-target earnings (OTE) and base compensation range for this role is USD $154,445–$185,334. Actual compensation may vary based on the candidate's level, experience, and skill set. Compensation ranges are country-specific, and candidates applying from other locations will be informed of the applicable market pay range and benefits by their recruiter. All roles include Restricted Stock Units (RSUs), giving team members ownership in Grafana Labs' success. The company also provides a global annual leave policy of 30 days per year, including three days reserved for Grafana Shutdown Days, subject to applicable local legislation. Other benefits and workplace practices include remote work, in-person onboarding, transparent communication, career development opportunities, and access to company-funded AI coding assistant usage within security guidelines. Equal Opportunity / EEO Grafana Labs is an equal opportunities employer. The company welcomes applications regardless of race, colour, nationality, origin, caste, sex, gender reassignment, gender identity or expression, sexual orientation, age, religion or belief, disability, veteran status, genetic information, pregnancy, maternity, marital, family or carer status, or any other characteristic protected by local law. Grafana Labs believes equality and diversity are essential to building a strong organization. Grafana Labs may use AI tools during recruitment to help match information provided in CVs with job postings. The recruitment team will continue to review applications manually to assess alignment with current openings.