OverviewIdentifies and understands problems facing projects and is able to leverage knowledge of data science to uncover important factors that can influence outcomes on specific products. Acquires data necessary for successful completion of the project plan. Leverages knowledge of modeling and machine learning solutions to identify the best approach to complete objectives. Understands relationship between technical metrics and business objectives. Uses business knowledge and technical expertise to provide feedback to the engineering team to identify potential future business opportunities. Writes efficient, readable, extensible code from scratch that spans multiple features/solutions. Collaborates with end customer and Microsoft internal cross-functional stakeholders to understand business needs. Applies a customer-oriented focus by understanding customer needs and perspectives, validating customer perspectives, and focusing on broader customer organization/context. Collaborates with relevant partners on defining metrics and frameworks for evaluating complex AI/LLM systems and agentic product performance.
ResponsibilitiesAI Operations
Collaborates with relevant partners on defining metrics and frameworks for evaluating complex AI/large language model (LLM) systems and agentic product performance. Diagnoses complex issues and identifies opportunities to drive improvements within AI/LLM systems within domain areas, acting as a resource for junior team members. Selects the appropriate model (e.g., LLM, prompting approach) for tasks. Prototypes and deploys applications of AI models using containerized environments (e.g., AI Azure Foundry, LangChain) to operationalize, build, and validate scalable tools/application/solutions for use by internal customers. Optimizes scalable systems that use optimal context engineering (e.g., retrieval-augmented generation) workflows, including the selection of data sources, retrieval systems, and generative models to ensure quality output. Drives AI system roadmaps through continual assessment of model performance and refinement of LLM systems and models.
Business Management
Collaborates with end customer and Microsoft internal cross-functional stakeholders to understand business needs. Formulates a roadmap of project activity that leads to measurable improvement in business performance metrics over time. Influences stakeholders to make solution improvements that yield business value by effectively making compelling cases through storytelling, visualizations, and other influencing tools. Exemplifies and enforces team standards related to bias, privacy, and ethics.
Business Understanding and Impact
Identifies and understands problems facing projects and is able to leverage knowledge of data science to uncover important factors that can influence outcomes on specific products. Produces a project plan to specify necessary steps required for completion. Assesses current situation for resources, risks, contingencies, requirements, assumptions, and constraints. Coaches less experienced engineers in data standards and best practices. Understands business strategy briefings and articulates data driver strategies for specific industries or cross-industry functions, such as: Sales/Marketing, Operations, and new Data Monetization Schemes. Engages business stakeholders to capture and shape their thinking on data-driven methods applicable to their value chain. Leads customer conversations to understand, define, and solve business problems and business workflows in products or solutions to identify gaps.
Coding and Debugging
Writes efficient, readable, extensible code from scratch that spans multiple features/solutions. Develops technical expertise in proper modeling, coding, and/or debugging techniques such as locating, isolating, and resolving errors and/or defects. Understands the causes of common defects and uses best practices in preventing them from occurring. Collaborates with other teams and leverages best practices from those teams into work of their own team. Mentors and guides less experienced engineers in better understanding coding and debugging best practices. Builds professional-grade documents for knowledge transfer and deployment of predictive analytic models. Leverages technical proficiency of big-data software engineering concepts, such as Hadoop Ecosystem, Apache Spark, continuous integration and continuous delivery (CI/CD), Docker, Delta Lake, MLflow, AML, and representational state transfer (REST) application programming interface (API) consumption/development.
Customer/Partner Orientation
Applies a customer-oriented focus by understanding customer needs and perspectives, validating customer perspectives, and focusing on broader customer organization/context. Promotes and ensures customer adoption by delivering model solutions and supporting relationships. Works with customers to overcome obstacles, develops tailored and practical solutions, and ensures proper execution. Builds trust with customers by leveraging interpretability and knowledge of Microsoft products and solutions. Helps drive realistic customer expectations, including information about the limitations of their data.
Data Preparation and Understanding
Acquires data necessary for successful completion of the project plan. Proactively detects changes in datasets or anticipated outcomes (e.g., product/consumer behavior) and communicates to senior leads or leaders. Develops usable data sets for modeling purposes by leveraging AI tools (e.g., Codex, GitHub Copilot) to identify gaps in current processes and improve data quality. Contributes to ethics and privacy policies related to collecting data, preparing data, and using AI tools (e.g., Codex, GitHub Copilot) by providing updates and suggestions around internal best practices. Contributes to data integrity/cleanliness conversations with customers.
Evaluating for Insight and Impact
Understands relationship between technical metrics and business objectives. Ensures clear linkage between selected models and desired business objectives. Evaluates and identifies the degree to which models and measures meet business objectives and desired solutions. Defines and designs feedback and evaluation methods. Coaches and mentors internal stakeholders (e.g., software engineering managers or technical program managers) as needed. Presents results and findings to senior internal and external stakeholders.
Industry and Research Knowledge / Opportunity Identification
Uses business knowledge and technical expertise to provide feedback to the engineering team to identify potential future business opportunities. Develops a better understanding of work being done on team, and the work of other teams to propose potential collaboration efforts. Coaches and provides support to teams to execute strategy. Leverages capabilities within existing systems. Shares knowledge of the industry through conferences, white papers, blog posts, etc. Researches and maintains deep knowledge of industry trends, technologies, and advances. Actively contributes to the body of thought leadership and intellectual property (IP) best practices.
Modeling and Statistical Analysis
Leverages knowledge of modeling and machine learning solutions (e.g., state-of-the-art statistical analysis, deep learning, causal modeling, reinforcement learning) to identify the best approach to complete objectives. Understands modeling techniques (e.g., validation, hypothesis testing, optimal use of data) and machine learning (e.g., supervised and unsupervised learning, estimation and forecasting) in order to select the correct approach to prepare data, train and optimize the model, and evaluate the output for statistical and business significance. Understands the risks of data leakage, the bias/variance tradeoff, methodological limitations, etc. Writes all necessary scripts in the appropriate language: T-SQL, U-SQL, KQL, Python, R, etc. Constructs hypotheses, designs controlled experiments, analyzes results using statistical tests, and communicates findings to business stakeholders. Effectively communicates with diverse audiences on data quality issues and initiatives. Understands operational considerations of model deployment, such as performance, scalability, monitoring, maintenance, integration into engineering production system, stability. Develops operational models that run at scale through partnership with engineering teams. Coaches less experienced engineers on data analysis and modeling best practices. Develops a strong understanding of the Microsoft toolset in artificial intelligence (AI) and machine learning (ML) (e.g., Azure Machine Learning, Azure Cognitive Services, Azure Databricks). Breaks down complex statistics and machine learning topics into manageable topics to explain to customers and other stakeholders. Provides guidance on model operationalization that is built into the project approach using existing technologies, products and solutions, as well as established patterns and practices. Understands and keeps up to date on embedding models and vector databases.
Required/minimum qualifications
- Master's Degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field
- 3+ years data-science experience (e.g., managing structured and unstructured data, applying statistical techniques and reporting results)
Additional or preferred qualifications
- Doctorate in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or related field
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
— Sarah J.
·
Marketing Manager
★★★★★
Verified
