back to work
Final Year Project

IHS

Intelligent Home Surveillance

View on GitHub

IHS (Intelligent Home Surveillance) is an end-to-end home-safety platform that combines deep-learning video analytics with a Flutter mobile companion app and a desktop control panel. It continuously monitors camera feeds to protect elderly individuals and children, detecting falls, recognizing hazards, watching for child-hazard proximity, and offering AI-powered first-aid guidance.

01 / System architecture

System architecture

Hover a component to see what it does.

REST · WebSocket (frames)
auth · data · push
FastAPIInference Engine

YOLOv8-Pose · TCN · Hazard YOLO · Heuristic FSM. Hosts DeviceMonitorManager, the RAG service, the Groq LLM, and the FCM sender.

02 / Fall-detection pipeline

Fall-detection pipeline

A hybrid of a learned temporal model and rule-based biomechanics, built for robustness, not just accuracy on a benchmark.

  1. 01
    Pose Estimation

    YOLOv8s-Pose extracts 17 COCO keypoints per person.

  2. 02
    Feature Engineering

    Hip-centred normalisation, joint angles, velocities, trunk tilt → a 61-dim feature vector.

  3. 03
    TCN Classifier

    3-layer residual TCN over 30-frame sliding windows, run every 10 frames.

  4. 04
    Heuristic FSM

    Rule-based state machine on trunk angle, hip height, CoG velocity, and inversion detection.

  5. 05
    Score Blend

    70% TCN + 30% heuristic, with gating for confirmed falls.

  6. 06
    Sticky Lock

    Hysteresis prevents rapid state oscillation between fall / no-fall.

03 / Capabilities

Capabilities

Fall Detection

Hybrid TCN + heuristic classifier blending temporal pose sequences with rule-based biomechanics.

Inactivity Alerts

Timer-based alarm when a person stays on the floor beyond a configurable threshold.

Hazard Detection

YOLOv8 (INT8 OpenVINO) detector for knives, fire, stairs, ovens, and stoves.

Child Safety

Skeleton-ratio age classification with proximity alerts when children approach hazards.

Background Monitoring

Server-side DeviceMonitorManager processes RTSP/HTTP streams independently of client connections.

First-Aid Chatbot

RAG Q&A over official first-aid manuals using FAISS + BM25 hybrid retrieval with a Groq LLM.

AI Incident Reports

LLM-generated caregiver summaries from aggregated detection history.

Push Notifications

Firebase Cloud Messaging alerts delivered even when the app is closed.

04 / Tech stack

Tech stack

Inference
PyTorchYOLOv8 (Ultralytics)OpenVINOOpenCVSciPy
Backend
FastAPIUvicornWebSocketsasyncio
RAG
FAISSBM25sentence-transformersGroq API
Database
Supabase (PostgreSQL + Auth + RLS)
Push
Firebase Cloud Messaging (HTTP v1)
Apps
Flutter (Mobile + Desktop/Web)Provider

Thanks for reading. Want to see the rest?

Back to all work