INTERNSHIP - Data engineering & AI-ready software architecture

About this role

The scientific problem Every week, NMH experiments produce large volumes of heterogeneous measurements: plate-reader kinetics, microbiological assay readouts, imaging data, each in the native format of the instrument that produced it. Today, turning that raw output into an analysable result is largely manual : Experimental time is lost to data wrangling rather than to designing and running the next experiment. Results are hard to compare across campaigns , because the same biological variable is recorded differently depending on the instrument, the operator, and the date.

Reprocessing a past experiment is difficult , which limits reproducibility and prevents the accumulated data from being reused to generate new hypotheses. Internship goal: build a web interface in streamlit, python layer that takes raw instrument output to a clean, described, queryable experimental result and make that result directly consumable by modern AI architectures (LLMs, RAG pipelines, autonomous agents), so that a scientist can interrogate their own data in natural language.

Key responsibilities Instrument data acquisition & parsing: Design automated extraction modules capable of ingesting raw experimental data streams from various laboratory instruments and heterogeneous file formats, ensuring cleaning, typing, and quality control. Data standardization & schema modeling: Define and implement strict, unified data schemas (metadata, type validation, JSON serialization formats) to ensure data interoperability and reproducibility.

Architecture: Structure datasets and develop tools/function calling endpoints, enabling AI agents or LLM/RAG pipelines to autonomously query, analyze, and manipulate standardized datasets. API & internal tooling: Expose processing services via REST APIs and build lightweight interfaces (e.g., Streamlit) enabling scientific teams to visualize, validate, and query data independently. Apply Biodata’s software engineering best practices: containerization (Docker), version control (Git), unit/integration testing, and technical documentation.

Expected deliverables By the end of the internship you will deliver a deployed, operational standardization pipeline for at least one NMH data family; a documented experimental data schema; an LLM‑queryable access layer; and a lightweight interface used by the scientific team. Success will be demonstrated via a live demo, automated tests, deployment instructions, a short technical report, and readiness for inclusion in the supported scientific campaign.