Skip to content

image_segmentation

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

image_segmentation

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
c++_segmentation
c++_segmentation
 
 
kotlin_cpu_gpu
kotlin_cpu_gpu
 
 
kotlin_npu
kotlin_npu
 
 
swift_litert
swift_litert
 
 
swift_tflite
swift_tflite
 
 
README.md
README.md
 
 

README.md

LiteRT Image Segmentation Samples

This directory contains Android image segmentation samples demonstrating how to use LiteRT (Google's new runtime for TensorFlow Lite) with different hardware accelerators. The samples perform multi-class image segmentation to identify and classify objects in images.

Overview

Image segmentation is a computer vision task that involves partitioning an image into multiple segments or regions, where each pixel is assigned to a specific class. These samples demonstrate real-time segmentation on Android devices using optimized neural network models.

Available Implementations

1. kotlin_cpu_gpu

A standard implementation supporting CPU and GPU acceleration for broad device compatibility.

Features:

  • CPU and GPU delegate support

  • Real-time camera and gallery image segmentation

  • 21-class segmentation (person, car, bicycle, etc.)

  • Compatible with a wide range of Android devices

Performance on Samsung S25 ULtra:

  • CPU: 120-140ms per frame

  • GPU: 40-50ms per frame

2. kotlin_npu

An advanced implementation with Neural Processing Unit (NPU) support for significantly faster inference on compatible devices.

Features:

  • CPU, GPU, and NPU delegate support

  • Optimized for Qualcomm and MediaTek NPU hardware

  • Same 21-class segmentation as CPU/GPU version

Performance on Samsung S25 Ultra:

  • CPU: 120-140ms per frame

  • GPU: 40-50ms per frame

  • NPU: 6-12ms per frame (10-20x faster than CPU!)

3. swift_litert (iOS)

An iOS SwiftUI implementation using the LiteRT CompiledModel Swift API (import LiteRT).

Features:

  • Native Swift Environment, CompiledModel, and TensorBuffer APIs
  • CPU (XNNPACK delegate) and GPU (LiteRtMetalAccelerator) backend switching
  • Real-time camera and gallery image segmentation

4. swift_tflite (iOS)

An iOS SwiftUI implementation using the TensorFlow Lite Interpreter Swift API (import TensorFlowLite) integrated into the LiteRT Swift Package.

Features:

  • Native Swift Interpreter, MetalDelegate, and CoreMLDelegate APIs
  • 3-way hardware backend switching: CPU (XNNPACK), GPU (MetalDelegate), and NPU / Apple Neural Engine (CoreMLDelegate)
  • Real-time camera and gallery image segmentation

Technical Details

Model Architecture

  • Input: 256x256x3 RGB image

  • Output: 256x256x6 segmentation mask

  • Model format: TensorFlow Lite (.tflite)

Key Dependencies

  • LiteRT (com.google.ai.edge.litert)

  • Android CameraX: For camera functionality

  • Jetpack Compose: For modern UI

  • Kotlin Coroutines: For asynchronous operations

Note

The LiteRT runtime and dependency versions in these samples are pinned as a baseline reference. When testing newer releases, nightly builds, or custom-built binaries, update the corresponding project configurations to match your requirements.

Architecture Components

  • ImageSegmentationHelper: Core segmentation logic and model management

  • MainActivity: UI orchestration with camera/gallery tabs

  • MainViewModel: State management and data flow

  • TensorUtils: Utilities for tensor inspection and debugging

Contributing

When contributing to these samples:

  1. Follow existing code style and patterns

  2. Test on multiple devices and accelerators

  3. Update documentation for any new features

  4. Include performance metrics for optimizations