Software Development Engineer in Test
About SOFICO Intelligence SOFICO Intelligence applies advanced AI and intelligence techniques to transform and extract value from the deep business controls embedded in back-office systems. We build agentic AI that reasons over the complex financial, leasing, fleet, and mobility-management data that runs the enterprise, surfacing fraud, billing errors, hidden risk, and value that traditional tools miss. We are backed by SOFICO, a global leader in ERP software for automotive finance, leasing, fleet, and mobility management, which gives us deep domain and back-office expertise to build on. Remote work is part of how we operate, and our team is made up of curious, self-directed people who like to build and ship. Our flagship AI platform, Velona, is a multi-agent system that turns years of business and operational data into financial intelligence. It finds fraud, ghost assets, billing errors, and risk on its own, and reports each finding as a specific asset, a dollar amount, and a recommended action. Rather than acting as a chatbot or a dashboard, it performs real analytical reasoning, runs investigations overnight without supervision, and answers in plain language. What the job is about This role is responsible for making our autonomous AI trustworthy. As an SDET on the AI team, you own quality for non-deterministic, multi-agent systems, which raises questions most testing roles never face. How do you prove an AI reached the right answer for the right reason? How do you catch a hallucination before a customer sees it? How do you guarantee one tenant can never see another's data, that an agent never exceeds its guard rails, and that quality does not quietly drift when a model or prompt changes? You will help define how SOFICO Intelligence evaluates agentic AI, designing and building the evaluation harness that underpins our AI governance and our confidence to ship. This is real engineering against a production AI system at scale, not script-running against a static application. You write code to test code, and code to assess the quality of the AI itself. It is one of the more interesting testing problems in software today, and you would own it. What you'll do (about 85% of the role) - Design and build SOFICO Intelligence's evaluation harness for agentic AI: golden datasets, automated scoring, faithfulness and grounding checks, and regression detection across prompt, model, and agent changes. - Verify grounding and faithfulness, confirming every insight and every number traces back to real data, and build the automated checks that stop hallucinations from reaching production. - Test multi-agent orchestration end to end: agentic tool-calling loops, supervisor-specialist pipelines, autonomous background jobs, and the synthesis of findings into trustworthy reports. - Red-team the system for prompt injection, jailbreaks, and tool misuse, and rigorously prove multi-tenant data isolation so one organization can never reach another's data. - Validate that AI guard rails hold under pressure: iteration limits, agent-recursion depth, and token/cost ceilings behave correctly at the edges. - Pressure-test performance and scale: real-time streaming, high-concurrency conversations, latency, and cost under realistic and adversarial load. - Build automated tests for the Go microservices, data/SQL tools, and APIs that power the platform, and wire them into CI/CD for continuous evaluation. - Partner with AI and data engineers to define quality bars and acceptance criteria, and help evolve the supervisor review-and-retry loop that keeps the system honest. The remaining ~15%, breadth across the platform - Strengthen automated testing across the broader SOFICO Intelligence platform and, where it intersects, the wider SOFICO engineering environment, bringing modern test practices to established systems and tooling. - Help raise quality engineering standards consistently across the combined group. Why this role matters - Hard, important work. Evaluating autonomous, reasoning