Own the science behind Alessia’s answers: retrieval, grounding, and evaluation on real manufacturer documentation (cut sheets, IOMs, material charts, and competitive guides), not generic web corpora.
About the role
Back Bay Automation builds Alessia, an AI assistant trained on each customer’s technical library so sales, applications, and field teams get spec-accurate answers in minutes. You will advance our core RAG stack: document understanding, chunking and indexing strategies, reranking, citation fidelity, and offline evaluation tied to customer success metrics.
This is a hands-on research role with weekly production impact. You will run experiments on real (permissioned) industrial corpora, publish internal benchmarks, and partner with data engineering and product on what ships to customers.
Responsibilities
- Design and improve retrieval pipelines for PDFs, scanned pages, tables, diagrams, and metadata-rich product data
- Build evaluation suites (faithfulness, citation accuracy, coverage, latency) using customer-shaped document sets and golden Q&A pairs
- Prototype and productionize embedding, reranking, and query-routing strategies across model providers
- Research grounding techniques that reduce hallucination on numeric specs, model numbers, and material compatibility claims
- Collaborate with data engineering on chunking policies, index versioning, and regression detection when libraries update
- Partner with product and customer success to translate failure modes into measurable research milestones
- Document findings, maintain experiment tracking, and present tradeoffs to engineering and leadership
- Stay current on LLM/RAG literature and selectively adopt techniques that fit our latency, cost, and accuracy constraints
Required qualifications
- MS or PhD in CS, ML, NLP, Information Retrieval, or equivalent industry experience (strong BS + 3+ years also considered)
- Demonstrated work with LLMs, embeddings, vector search, or semantic retrieval systems
- Strong Python skills and comfort with notebooks, experiment tooling, and reproducible analysis
- Experience designing offline metrics and human-in-the-loop eval workflows
- Ability to read research papers and implement practical variants under resource constraints
- Clear written communication for technical specs and experiment reports
Preferred qualifications
- Experience with document AI: layout parsing, OCR post-processing, table extraction, or multimodal PDF understanding
- Background in industrial, manufacturing, or technical B2B domains
- Familiarity with LangChain/LlamaIndex-style stacks, vLLM, or similar production LLM tooling
- Publications, open-source contributions, or Kaggle-style competition results in NLP/IR
- Exposure to prompt optimization, fine-tuning, or distillation for domain-specific assistants
What we offer
- Competitive salary and equity participation in an early-stage, customer-funded product company
- High ownership: your research directly shapes answer quality for instrumentation, manufacturing, and distribution customers
- Hybrid work with flexible hours; core collaboration windows with the NYC-area team
- Learning budget for conferences, courses, and tooling
- Health benefits and paid time off (details shared at offer stage)
Back Bay Automation is an equal opportunity employer. We welcome applicants from all backgrounds and do not discriminate on the basis of race, religion, gender identity, sexual orientation, age, disability, or veteran status.