Jorge Ortiz
Sensing & Reasoning
Building perception-to-decision systems with calibrated competence.
How should an AI system decide when evidence is insufficient?
I care about making AI systems that work in the real world and high-stakes environments. I connect formal methods, runtime monitoring, and multimodal sensing to study when a deployed system should act, when it should defer, and what evidence it should preserve for audit. That work spans agentic workflows across trust boundaries, perception pipelines that judge their own competence, and sensing infrastructure where privacy and accuracy trade off.
The diagram above is how I decompose the problem, with explicit competency boundaries as a first-class output. I pursue this across agentic systems, urban sensing, human-machine systems, and instrumented health spaces, detailed on the Lab site and below. Before Rutgers I worked at IBM Research and in several startups. MS and PhD from UC Berkeley, BS from MIT.
Research Program
We build AI systems that must be trustworthy under real operating conditions, from agentic workflows that need formal guarantees to sensing systems that must know when to abstain.
Verified Agentic AI
Agentic AI systems call tools, coordinate tasks, share resources, and act across trust boundaries. When coordination fails, the result is deadlocks, races, and silent protocol violations. We study how to make these systems formally verifiable before deployment and auditable during execution.
TraceFix (Best Paper, ACM CAIS 2026) uses TLA+ model checking to verify coordination protocols and counterexample-driven repair to fix them. Verified protocols are enforced at runtime by a topology monitor. We are extending this to guardrails for harness drift, evidence grounding, and post-training governance for deployed agents.
Urban AI
Urban environments generate a lot of sensor data, but data alone can't tell correlation from causation. We work on causal discovery and privacy-preserving sensing so decision-makers can predict what happens under interventions.
TeLLMe does causal discovery from urban video with self-confidence assessment; we've run it on real dashcam data with a causal-card interface for policy-style questions. CityOS builds on a formal impossibility result (privacy and correct counting fail for the same reason, ambiguity) and is piloted for parking, assistive crossing, and crowd-density at CS3 testbeds in NYC, Rutgers, and West Palm Beach.
Social Reasoning in Human–Machine Systems
How can robots read social context when that takes common sense we don't know how to program? And when are those inferences trustworthy enough to act on?
We work on VLM-based social HRI (nonverbal cues), SoNNET for bite timing in robot-assisted feeding, including group dining (CoRL 2022), and Project Paz for driver attention and when to interrupt. CAMERA (NIH BRAIN Initiative, $5M) targets multimodal behavioral biomarkers for anxiety. Collaboration with Cornell on caregiving robotics; published at CoRL and IMWUT.
Multimodal Spatial Reasoning
Instrumented spaces give you many modalities at once. The hard part is what a space affords and how to handle distribution shift, same kind of event, different readings across time and place.
DFGauss (NeurIPS 2025) does 3D occupancy prediction; Maestro is our 18-channel multisensor platform deployed across multiple campus buildings.
Future Directions
I'm working toward systems that discover causal, social, and spatial structure, know where that structure applies, and can say so clearly to the people and organizations that rely on them. That means formal ways to represent when conclusions hold across tasks, methods for systems to learn their own boundaries from experience and feedback, and interfaces that make those boundaries interpretable.
This vision drives the next 5–10 years of my work through CRAIG (governance), CS3 (urban deployment), and NIH CAMERA (health sensing), three funded initiatives where knowing when to trust the system is central.
Selected Publications
Full list on Google Scholar.
Media & Press
The Overestimation of Artificial Superintelligence
Interviewed about AI capabilities and the hype around superintelligence
Robot that learns social cues could feed people with tetraplegia
Coverage of our CoRL 2022 work on bite timing prediction for robot-assisted feeding in group dining settings
News
Centers & Initiatives
Different scales, governance, city infrastructure, the lab, and applied work.
CRAIG. Responsible AI & Governance
Role. Rutgers Site Director
When should AI systems act? We develop theoretical foundations and evaluation pipelines for systems that know their boundaries.
CS3. Smart Streetscapes
Role. Rutgers Site Lead
City-scale sensing and causal reasoning for pedestrian safety. We deploy and evaluate urban AI in the real world.
Sensing & Reasoning Lab
Role. Director
Perception, causal inference, and self-assessment, the core work that feeds the three pillars above.
New York Yankees
Role. Research Analyst
Computer vision and biomechanics from video for player development, strategy, and performance, systems that have to signal when they're uncertain.
Teaching
Selected courses
ECE 252. Programming Methodology I
Introduction to C and C++ (100–170 students, taught six times). Covers language expressivity, OS memory models with a pictorial representation I devised, and object-oriented design: students develop OO strategies using only C data structures. I work backward from practice to theory, framing recursion through web crawlers before presenting the generalization.
ECE 532. Multimodal Learning for Sensing Systems
Multimodal learning across audio, video, time series, and text for sensing systems. Students work on real data and deployed settings; capstone teams build systems that run in the lab and in the field.