Read-Only Mode
AU

Livello Smart-Store-System — Project Overview

View: Feb 2020 – Dec 2023
Project ran February 2020 – end of 2025 · view window shows Feb 2020 – Dec 2023|8 Streams · Software · Hardware · Computer Vision
2,132
Tickets with Work Logs
106
Team Members
8
Streams
47 months
View Window (Feb 2020 – Dec 2023)

Project Description

Expand

Project Goals

1.Develop a modular, IoT-based smart kiosk platform capable of autonomous product recognition and automatic billing — without requiring staff interaction at the point of sale.
2.Achieve reliable product identification through sensor fusion: combining precision weight measurement (load-cell arrays) with computer vision (camera-based tracking) to handle multi-product, multi-user scenarios in real-world retail conditions.
3.Build a cloud-based Mission Control platform for real-time inventory monitoring, remote diagnostics, and operational analytics — enabling centralized management of a distributed kiosk fleet.
4.Design and validate production-ready hardware: modular smart shelves with integrated sensors, thermal management for refrigerated/frozen goods, and a scalable kiosk enclosure supporting iterative field deployment.
5.Enable contactless, multi-user shopping experiences through a consumer mobile app, kiosk touch interface, and QR-based access — with seamless payment terminal integration.
6.Evaluate the feasibility of scaling from single-kiosk autonomy to a full smart-store environment with simultaneous multi-user tracking and person-to-product attribution via computer vision.
7.Establish systematic testing and validation workflows — from lab-based integration sandboxes through pilot field trials — to measure system reliability under real-world operating conditions before production rollout.
8.Implement security-by-design across all layers: device identity and provisioning, role-based access control, secure OTA updates, and audit logging — meeting operational security requirements for unattended retail environments.

Planning Assumptions & Unknowns

Collapse
Assumptions
A1Load-cell sensors can achieve sufficient resolution and repeatability to detect individual product pick/put events on a shared shelf — if temperature drift and mechanical vibration are compensated systematically.
A2Edge compute hardware (Jetson-class devices) will provide adequate inference throughput for real-time person detection and tracking at acceptable cost and power envelope for a retail kiosk form factor.
A3A cloud-based Mission Control architecture can reliably manage a distributed fleet of kiosks with intermittent connectivity — requiring robust offline buffering and eventual-consistency reconciliation on the edge.
A4Modular hardware design (shelf units, sensor boards, enclosures) will allow iterative field testing without full production tooling — enabling rapid prototype-test-learn cycles within the project timeline.
A5Existing authentication providers (Auth0) and payment terminal APIs can be integrated to meet the security and transaction requirements of unattended retail — without building custom identity or payment infrastructure.
A6A single development team spanning hardware, firmware, cloud, and CV disciplines can maintain sufficient integration velocity across all subsystems through shared tooling (Jira, CI/CD, Mission Control).
A7A single scale and electronics architecture can be generalized to support multiple kiosk types (fridge, freezer, ambient) — avoiding parallel hardware lineages.
A8One common embedded/edge platform can coordinate all local device functions (sensors, locks, displays, payment, vision, networking) without per-device custom stacks.
A9Third-party integrations for payment, ERP, and identity can be made sufficiently robust through standard APIs and adapter layers — without bespoke per-customer integration code.
Unknowns & Open Questions
U1Can weight sensing alone achieve production-grade product recognition and fraud prevention (>99% accuracy) across varying weights, shelf loads, and environmental conditions — or is sensor fusion mandatory?
🔶 PartialWeight-only is sufficient at single-kiosk scale with per-shelf calibration. Store-scale required additional signals — a sensor-fusion approach (weight + CV + session context) emerged as the minimum viable combination.
U2Can computer vision reliably attribute product interactions to specific individuals in a multi-user, open-store environment with occlusions, lighting variation, and diverse body types — using only edge inference?
⚠️ Not achievedNo. Store-level multi-user attribution did not reach the reliability threshold required for autonomous billing. Single-kiosk CV (person counting, demographics) validated successfully.
U3What is the minimum sensor density (load cells per shelf, cameras per zone) needed to achieve reliable product-level event detection without unacceptable false-positive rates?
🔶 PartialEstablished for kiosk-scale (4–6 shelves). Store-scale sensor density requirements remain open.
U4Will the thermal management system (compressor + fan + insulation) maintain safe product temperatures across the full operating range without exceeding the kiosk’s power and noise budget?
ResolvedYes, validated through extended field trials with continuous temperature logging.
U5Can the system handle graceful degradation when individual components fail (sensor board offline, network drop, CV pipeline crash) without disrupting active customer sessions?
🔶 PartialPartially. Edge buffering and session recovery implemented. Full multi-component failure recovery not tested at scale.
U6What regulatory and compliance requirements (CE, EMC, electrical safety, GDPR) apply to an unattended retail IoT device — and how early must these be addressed to avoid costly late-stage redesigns?
ResolvedCE/EMC testing completed at kiosk level. GDPR review conducted for camera-based systems. Full store-level compliance not pursued.
U7Can the weighing concept remain stable under fridge and freezer conditions — i.e. across sustained low temperatures, condensation, and door-cycle thermal swings?
🔶 PartialYes for refrigerated kiosks after compensation logic; freezer-grade stability required additional mechanical and firmware adaptations.
U8Is pseudo-absolute or delta-based weighing logic sufficient to avoid repeated manual recalibration during normal operation?
🔶 PartialDelta-based logic with periodic auto-zero proved sufficient under typical usage; outlier conditions still required occasional manual recalibration.
U9Are the chosen PCB architecture, ADC, microcontroller, and firmware model sufficient to meet measurement, control, and communication requirements across all kiosk variants?
ResolvedThe selected platform met requirements for the validated kiosk variants; longer-term scaling to additional variants would require an architecture review.
U10Can the mechanical design achieve repeatable sensor behavior despite assembly tolerances and field handling?
🔶 PartialAchieved through fixturing, calibration jigs, and assembly procedures; remained sensitive to operator skill and incoming part quality.
U11Can the door, lock, and session mechanics be made fail-safe across reboot, brownout, network loss, and power cycling — without trapping customers or losing transactions?
🔶 PartialImplemented and validated in lab and pilot conditions; rare edge cases (simultaneous power + network loss mid-session) remained an open risk.
U12Can kiosks be remotely updated, debugged, and recovered safely at scale (including OTA firmware/software updates) without requiring on-site intervention?
🔶 PartialOTA, remote debug, and recovery flows implemented for the deployed fleet; large-scale rollout policies and rollback strategies were proven in operations but not stress-tested at hundreds of devices.
U13Is computer vision required only as an assistive signal (e.g. anti-fraud hints, demographics) or as a core recognition layer for billing?
🔶 PartialCV proved viable as an assistive signal at kiosk scale; as a core multi-user attribution layer at store scale, it did not meet the required reliability bar.

Stream Overview

#StreamPeriodTicketsHoursStatus
1
Concept & System Architecture
02/2020 – 12/2020942,952hCompleted
2
Sensor Calibration & Fusion
01/2021 – 11/20229015,537hCompleted (single-kiosk)
3
Cloud Backend & Mission Control
05/2020 – 12/202365326,595hCompleted
4
Hardware / Electronics / Mechanics
04/2020 – 12/202338127,821hCompleted
5
Kiosk UI, Access & Apps
04/2021 – 10/202355625,220hCompleted
6
Computer Vision & Tracking
01/2023 – 12/20231279,595h⚠️Store-level not achieved
7
Testing, Validation & Field Trials
06/2022 – 12/2023883,674hCompleted
8
Security, Identity & Compliance
01/2023 – 12/20231437,526h🔶Partial (kiosk-level)
Total2,132118,920h

Project Timeline (Gantt)

Stream / Task
1. Concept & System Architecture
2. Sensor Calibration & Fusion
3. Cloud Backend & Mission Control
4. Hardware / Electronics / Mechanics
5. Kiosk UI, Access & Apps
6. Computer Vision & Tracking
7. Testing, Validation & Field Trials
8. Security, Identity & Compliance
2020Jan
Mar
Jun
Sep
Dec
2021Jan
Mar
Jun
Sep
Dec
2022Jan
Mar
Jun
Sep
Dec
2023Jan
Mar
Jun
Sep
Dec