Neri Carcasci

Red Hat

Red Hat

2025, 2026 to now ยท 10 mos

AI Associate Engineer, MLflow

Open-source MLflow, working on how agent runs are recorded and how evaluations are executed.

  • Supporting open-source development of MLflow with a focus on agent observability: trace archival and streaming retrieval, evaluation job execution, and the communication between the server and the client SDK.
  • Designed a pluggable execution framework for evaluation and scoring jobs, with process isolation, cancellation and timeout semantics held under test.
  • Partnering with other Red Hat product teams to land their MLflow integrations, translating their requirements into upstream changes and carrying them through public review.
  • Maintaining Red Hat's downstream MLflow distribution shipped in OpenShift AI across supported release streams: release engineering, CI stability and dependency remediation.

Python / FastAPI / ASGI and async I/O / MLflow / agent tracing / OpenShift AI / Kubernetes / pytest / GitHub Actions

  • Built the LLM evaluation metrics surface for TrustyAI: BLEU, ROUGE and Levenshtein.
  • Rewrote the mathematical foundations of the drift and anomaly detection suite, including a full Python implementation of Jensen-Shannon divergence, and designed the API layer around it.
  • Architected the service test infrastructure using dependency injection, with validation coverage across every endpoint.

Python / NumPy / SciPy / statistical testing / LLM safety and evaluation metrics / drift detection / dependency injection / Podman / Quay.io

All work

Neri Carcasci. 2026. Dublin.