Cohere Machine Learning Engineer Interview Questions

I would compare it against a baseline using task quality, robustness, latency, cost, and user satisfaction. For an LLM, I would also test refusal behavior, consistency, and behavior on edge cases because average scores can hide serious failure modes. A production-ready model is one that is useful and predictable in real usage.

I would start with prompt engineering because it is faster to iterate and easier to change. If the model still does not behave consistently enough, or if the task requires strong control over style or format, then fine-tuning becomes more attractive. The decision depends on the amount of control, data, and operational overhead I can justify.

I would compare the data pipeline, prompt format, retrieval settings, model version, and runtime configuration before and after the change. Then I would reproduce the issue on a fixed evaluation set and check whether the problem comes from data drift, a code regression, or a change in traffic. Debugging ML systems means treating the full stack as the product.

I would track quality, latency, throughput, error rate, drift, and cost per request. For generative systems, I would also watch hallucination rate, refusal behavior, and user feedback. The exact metrics should reflect both the model and the production system around it.

I would first estimate how much noise exists and whether it affects the business goal. Then I might clean the labels, down-weight uncertain examples, or use robust training methods depending on the task. Noisy labels are common, so the goal is usually to reduce their impact rather than pretend they do not exist.