AI/ML Engineer

About the Role 

We are building Joblogic’s AI Agent Platform — a multi-tenant system for designing, versioning, evaluating, and running AI agents that work across email, voice, SMS, WhatsApp, and CRM channels on behalf of our customers. The platform is built on a LangGraph runtime with a full agent lifecycle: a prompt-driven agent builder, a tool and knowledge-base registry, human-in-the-loop review, an agent memory subsystem, and a rigorous evaluation harness backed by LangSmith and PromptFoo. 

We are looking for a mid-level AI/ML Engineer to help us design, build, and continuously improve these agents. In this role you will own agent behaviour end to end — crafting and engineering prompts, wiring up tools and retrieval, and, most importantly, building the evaluations and datasets that prove an agent is doing the right thing before and after it ships. You will also bring applied machine learning depth: working with our execution and conversation data, building models and analyses that make agents smarter, and using platforms such as Databricks or AWS SageMaker to train, track, and serve them. 

You will work closely with product, backend, and platform engineers, and your work will directly shape how tens of thousands of field-service businesses experience intelligent automation. 

What You’ll Do 

• Build and improve AI agents — design agent behaviour on our LangGraph runtime, configure abilities, attach tools and knowledge bases, and take agents from draft through evaluation to production deployment. 

• Prompt engineering — author, version, and systematically improve system prompts (managed via our LangSmith prompt registry), applying techniques such as few-shot examples, structured output, tool-use guidance, and context management to get reliable behaviour from LLMs. 

• Design and run evaluations — build datasets, author heuristic and LLM-judge rubrics, run offline evaluations and online scoring against live executions, set pass/fail thresholds, and track quality and regression/drift over time. This is central to the role. 

• Retrieval & knowledge (RAG) — build and tune retrieval-augmented generation over our knowledge bases — chunking, embeddings, and hybrid/vector search — to keep agent answers grounded and faithful. 

• Tools & integrations — develop and integrate the tools agents call (HTTP, MCP, and recipe/workflow tools), extending what agents can do across our own and third-party systems. 

• Data analysis — analyse execution traces, conversation transcripts, and outcome data to find failure modes, quantify quality, and drive prioritised improvements. 

• Applied ML — build, train, evaluate, and deploy machine learning models (e.g. classification, extraction, ranking, forecasting) that support agent behaviour and the wider AI-first roadmap, including predictive maintenance and data-driven decisioning. 

• MLOps — use platforms such as Databricks or AWS SageMaker for feature engineering, experiment tracking, model training, and serving, with reproducible pipelines. 

• Observability & quality — use tracing and monitoring (LangSmith, Application Insights) to debug agent runs in production and close the loop from live behaviour back into datasets and prompts. 

• Collaborate & ship — work in a cross-functional team using tools such as Jira and Slack, write clear documentation, and ship iteratively with a strong quality bar. 

Essential Experience and Skills 

• Strong Python engineering skills, with experience building production services and clean, well-tested code. 

• Hands-on experience building LLM-powered applications or AI agents — for example with LangChain / LangGraph, the OpenAI / Anthropic APIs, or comparable frameworks — including tool/function calling and agentic (ReAct-style) loops. 

• Demonstrable prompt engineering skill: designing, iterating on, and versioning prompts, and understanding how to steer and constrain model behaviour reliably. 

• Practical experience with LLM evaluation: building datasets, defining rubrics/metrics (heuristic and LLM-as-judge), running evals, and interpreting results to improve a system. This is a core requirement. 

• A solid machine learning foundation: framing problems, feature engineering, training and evaluating models, and understanding metrics, validation, and overfitting — with hands-on use of libraries such as scikit-learn, PyTorch, or TensorFlow. 

• Hands-on experience with a modern ML platform — Databricks or AWS SageMaker (or equivalent, e.g. Azure ML / Vertex AI) — for training, experiment tracking, and model deployment. 

• Strong data analysis skills using Pandas, NumPy, and SQL to explore data, quantify behaviour, and communicate findings. 

• Understanding of retrieval-augmented generation (RAG): embeddings, vector/hybrid search, chunking, and grounding/faithfulness. 

• Experience working with APIs, JSON schemas, and integrating third-party services from Python. 

• Familiarity with both SQL and NoSQL databases. 

• Committed to continuous learning, proactive problem-solving, and timely issue identification, with a keen interest in staying current with a fast-moving field. 

• Strong communicator, experienced in collaborating with cross-functional teams using tools such as Jira and Slack. 

• Creative and innovative thinker, consistently contributing fresh ideas and solutions in alignment with current technological trends. 

Nice to Have 

• Experience with LangGraph and LangSmith specifically (runtime, tracing, prompt registry, and evaluations). 

• Experience with evaluation tooling such as PromptFoo. 

• Experience with vector search / vector databases (e.g. Azure AI Search, pgvector, Pinecone, or similar). 

• Experience with the Model Context Protocol (MCP) or building tool/agent integrations. 

• Experience with Python web frameworks (FastAPI / Flask) and asynchronous programming. 

• Experience with task queues / distributed workers (e.g. Celery) and event-driven architectures. 

• Experience deploying to a major cloud (Azure, AWS, or GCP), plus Docker and CI/CD. 

• Exposure to voice/telephony, email, or messaging integrations (e.g. Twilio, ElevenLabs, Nylas). 

• Awareness of AI safety, guardrails, evaluation of harmful outputs, and responsible-AI practices. 

• Experience with data pipelines and feature stores. 

What We Offer 

• Professional Working environment 

• Market Competitive Salary 

• Life Insurance & Medical Insurance (Including Family) 

• OPD 

• Provident Fund 

• Gym Facility 

• Maximum 45 Weekly Hours (Monday–Friday) 

• Remote Working (During Pandemic Situation) 

• Company trip 

• 29 Annual Leaves 

• 8 Sick & uncapped Compassionate Leaves (As per Company Policy)