Twitter / X Facebook LinkedIn WhatsApp Email Copy link

Report this job





Incorrect company
Incorrect location
Job is expired
Job may be a scam
Other





  Search Jobvertise
Evals Researcher, Rewards and Environments
Location: US-CA-San Francisco
Jobcode: 975186811
| Report Job

Apply Online  

About the Role: Evals Researcher, Rewards and Environments, San Francisco (hybrid)

Post-training and evaluation lab, profitable at four months, San Francisco

You would build the evaluation systems that decide which data is worth training on and whether the models and agents are actually getting better.


About the Company

Our company builds theevaluation environments and post-training infrastructure that make long-horizon enterprise agents trainable, then uses that stack to train and deploy custom models for very large technology companies and government customers. Under four months in, ten people, already profitable, on angel and institutional money and no priced round.


What you would do:

- Rank training value before compute is spent: work out which tasks in a dataset are worth training on and build a system that ranks every task by expected training value.

- Build scoring that runs constantly and cheaply, tuned to each customer's definition of a good outcome rather than generic correctness.

- Catch drift before it is a problem: notice when real customer requests have moved far enough from the test set that it no longer describes the job, and rebuild the evals to match.

- Design rewards: checkers a model cannot game, model-as-judge and rubric graders, and evaluation of long-horizon agent trajectories.

- Sit at the centre of the post-training loop: decide before compute is spent, and say afterwards whether a change actually helped.


Hard requirements from the JD

  • Built RL data or done RL research hands-on, not adjacent

  • Built an evaluation system themselves (graders, model-as-judge, environments or RL pipelines) that ran on production traffic or behind a published result

  • Real opinions on reward design: what makes a checker trustworthy and how models game it

  • Strong systems engineer who can pick up any stack; an engineering title on the current role

  • At least one year full-time, one to five typical

  • Six days of work a week, hybrid in San Francisco (three or four in the office)


Fill in the form, we will contact you...



Apply Online  
 
Search millions of jobs

Jobseekers
Employers
Company

Jobs by Title | Resumes by Title | Top Job Searches
Privacy | Terms of Use


* Free services are subject to limitations