Skip to main content

Rodrigo Monteiro Pereira · Data Platform & AI Lead Engineer

I build and govern the data platform that makes your AI work in production.

Data engineering and data science at the core; AI and agents as the differentiator — built on a governed platform, measured, and gated in CI.

About

Rodrigo Monteiro Pereira, Data Platform & AI Lead Engineer

I lead a Data & AI function end to end — architecture through execution. I started in C++ systems work in 2018, moved into data in 2021 and have led data teams since 2022, first in the financial market and today in telecoms, across on-prem and cloud.

My focus is the load-bearing part: medallion architecture, data quality, lineage, cost and access. AI and agents sit on top of that platform — with real evaluation suites, guardrails and CI regression gates, not demos.

Current focus

  • Governed lakehouse platforms — medallion, contracts, lineage
  • RAG and agents with evaluation suites and CI regression gates
  • MLOps — drift monitoring, leakage control, gated retraining
  • Governance under real regulators — LGPD, ANATEL, FCC, ANACOM
  • Data quality, master data and cost — the unglamorous half

Portfolio projects

Four projects on one shared open foundation. Statuses below are honest — nothing here is described as finished before it is.

  • Data platform · Lakehouse

    Status: Live

    Open Finance LakeHouse

    51 financial series, batch and streaming, on an entirely open stack.

    The foundation the rest of the portfolio is built on. Public Brazilian macro and market data lands as Delta on MinIO, is conformed by Spark into a star schema and served as DuckDB marts — every lane generated from one source registry. Adding a series is a YAML entry, not new code.

    • 51 registered series across 10 source handlers (BACEN, IPEA, IBGE, Tesouro, ANBIMA, B3, Yahoo Finance) — medallion bronze → silver → gold on Delta Lake over MinIO, ending in 8 BI-ready marts.
    • Airflow 3 Asset scheduling derived from the registry at parse time — 13 DAGs and 51 bronze assets, one task per series, so a failing API withholds exactly one asset and raises exactly one attributable alert.
    • Spark
    • Structured Streaming
    • Delta Lake
    • dbt
    • Airflow 3
    • MinIO
    • Polars
    • DuckDB
    • Pandera
    • OpenLineage
    • OpenMetadata
    • Kubernetes

    The Kubernetes/GitOps layer that deploys this stack lives in a separate repository that holds cluster credentials. It stays private permanently and is not linked from here.

  • Applied AI · Retrieval, evaluation & guardrails

    Status: In progress

    RAG over BACEN Copom minutes

    A retrieval defect, found by measurement and fixed by 120 lines of regex.

    RAG over 30 public minutes of the Copom, Brazil's monetary policy committee, built around an evaluation harness rather than a demo. Thirty documents each contain a near-identical decision paragraph, so dense retrieval confidently returned the right paragraph from the wrong meeting. Seven measured retrieval arms later, that defect is gone — and the component that fixed it was not the one anybody would have bet on.

    • The defect, measured: the dense baseline scored MRR 0.191 and got the right meeting at rank 1 on 4 of the 41 questions that name their meeting outright. It was not confused about the topic — it was answering a question nobody asked.
    • The fix, measured on the same question set: MRR 0.191 → 0.741, hit@5 0.959, rank-1 meeting correctness 0.098 → 1.000. It came from ~120 lines of regex that read the date out of the question and compile it into a Qdrant payload filter, at 41/41 hint precision and negative latency cost.
    • Python
    • Qdrant
    • bge-m3
    • BM25
    • cross-encoder
    • Ollama
    • Presidio
    • Langfuse
    • FastAPI
    • DuckDB

    Every number above comes from a committed JSON report in the repo. The gold set behind them is 56 draft pairs, not human-validated — so read the movement between arms, not the absolute level.

  • Machine learning · MLOps

    Status: In progress

    Hourly energy demand forecasting & drift

    The forecast is the easy half; the loop that decides when to retrain is the project.

    Hourly electricity demand forecasting for a US balancing authority, wrapped in the loop that actually matters in production: re-score a frozen model against the actuals that arrived, compute four kinds of drift, decide whether to retrain, and commit the numbers back to the repo. The code is real and tested end to end. The demand series is not — it is a seeded synthetic fixture pending an API key, and every artifact says so.

    • The demand series is a seeded synthetic fixture, because the free EIA API key has not been registered yet. Every metric artifact carries "is_real": false, the dashboard leads with a red banner, the PNGs are watermarked, and the cron refuses to publish without the key. No number from this project is a benchmark or a result.
    • Walk-forward backtesting with no temporal leakage: 56 daily folds × 24 horizons, identical folds and protocol for every model, with the no-leakage property enforced independently in the backtest and in the feature builder — and pinned by tests that poison future values and assert nothing moves.
    • Python
    • LightGBM
    • MLflow
    • FastAPI
    • pandas
    • Parquet
    • Evidently
    • React
    • ECharts
    • GitHub Actions

    Data caveat, stated up front: the demand series is synthetic pending a free EIA API key. The code is real; the numbers it produces describe a seeded curve, not a power grid, so none of them appear on this page as a forecast-accuracy result.

  • Master data management

    Status: Planned

    MDM / entity resolution

    One customer, five systems, no agreement — resolved into a golden record.

    Entity resolution over dirty, overlapping public datasets: blocking to make the comparison tractable, then ML and embedding based matching, ending in a single governed golden record. Designed to run on top of the Open Finance LakeHouse rather than as a separate stack. Not started — this card is a plan, and it says so.

    • Blocking strategy first, so the pairwise comparison stays tractable at scale.
    • Matching by ML and embeddings rather than hand-tuned string rules alone.
    • Python
    • Spark
    • Embeddings
    • Delta Lake
    • DuckDB

    Not started — no code yet

Other repositories

Smaller tools from the same toolbox

Experience

  1. Data Platform & AI Lead Engineer

    Feb 2026 – Present
    TelecallRio de Janeiro, Brazil

    Owns the Data & AI function end to end — architecture through execution — reporting directly to the IT Director, across an analytics estate spanning terabytes and three regulatory jurisdictions.

    • Databricks
    • Spark
    • Data governance
    • AI agents
    • GitOps
  2. Data & AI Consultant

    Aug 2023 – Feb 2026
    Consultoria IndividualRio de Janeiro, Brazil

    Independent data and AI consulting, between the Faros and Telecall roles.

    • Data Team Lead

      Jul 2022 – Aug 2023
      Faros Private InvestimentosRio de Janeiro, Brazil

      Led the data team, designing and shipping scalable data infrastructure for a private investment firm.

      • Databricks
      • Airflow
      • Spark
      • LakeFS
      • Python
    • Data Scientist

      Aug 2021 – Jul 2022
      Faros Private InvestimentosRio de Janeiro, Brazil

      Built predictive models and analytics products to sharpen commercial strategy and improve the client experience.

      • Python
      • PostgreSQL
      • Power BI
      • React
      • Next.js
    • Software Developer (Intern)

      Aug 2018 – Aug 2020
      Instituto TecgrafRio de Janeiro, Brazil

      Built modelling and simulation software for oil reservoirs in C++, OpenGL and Qt.

      • C++
      • OpenGL
      • Qt
      • 3D modelling
      • Simulation

    Skills

    Platform & governance

    • Apache Spark
    • Delta Lake
    • Apache Airflow
    • Databricks
    • LakeFS
    • MinIO / S3
    • Medallion architecture
    • Pandera
    • Great Expectations
    • OpenLineage
    • OpenMetadata

    Data science & ML

    • Python
    • pandas
    • Polars
    • scikit-learn
    • MLflow
    • Statistical modelling
    • Time-series validation
    • Drift monitoring (PSI / KS)

    AI & agents

    • RAG
    • Hybrid retrieval
    • Reranking
    • Evaluation suites
    • LLM-as-judge
    • Guardrails
    • Tracing (Langfuse)
    • Text-to-SQL
    • MCP

    Query & storage

    • SQL
    • DuckDB
    • PostgreSQL
    • Parquet
    • Power BI

    Cloud & DevOps

    • Azure
    • AWS
    • Docker
    • Kubernetes
    • FluxCD
    • Talos
    • Terraform
    • GitHub Actions
    • CI/CD

    Languages & interfaces

    • Python
    • C++
    • TypeScript
    • React
    • Next.js
    • Tailwind CSS
    • Recharts

    Background

    MSc in Data Science and Artificial Intelligence

    Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio)

    Aug 2025 – in progress

    BSc in Computer Science

    Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio)

    2016 – 2021

    Spoken languages

    • Portuguese — native
    • English — fluent
    • Spanish — fluent

    Get in touch

    Open to conversations about data platform, governance and applied AI roles.