Benchmarks

FedMABench

A federated mobile-agent training and evaluation suite that varies user-data heterogeneity and application distribution, with released partition annotations and inference/scoring scripts.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

Original source ↗Read as Markdown Record JSON

What is this resource?

Resource type: Benchmarks
Environment: android
Topics: evaluation, imitation-learning, mobile-navigation
Record publication date: Not recorded
Evidence retrieved: 2026-10-17T11:18:17Z
Resource ID: github:wwh0411/fedmabench

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code: Not recorded
Data: Not recorded
Document: Not recorded
Weights: Not recorded

Evaluation protocol

Focus: mobile-agent federated training, data heterogeneity
Setting: offline
Observations: mobile-agent examples with externally sourced screenshots
Actions / outputs: model predictions on mobile-agent examples
Evaluation: The provided evaluation workflow performs model inference over selected partitions and invokes test_swift_fed.py for accuracy scoring.

Scale and counting units

6 dataset families — released FedMABench partition families source
3 heterogeneity types — publisher benchmark overview source
8 federated learning algorithms — publisher benchmark overview source

The dataset-family count does not count tasks, apps, screenshots or client instances. Code and data licenses are unknown; external image sources are required.

Compare source-reviewed profiles

Recorded specifications

availability: Public federated-training/evaluation code and ungated annotation partitions; original images are separate dependencies.
repository revision: 417d8b0006830f423c7d07c213fc57277d75d966

Explore related topics

Evaluation Imitation learning Mobile navigation

Scope and limitations

The public Hub lists six dataset families of JSON/JSONL partition annotations. The repository explicitly says unpacked screenshots are not provided and must be reconstructed from original AndroidControl and AITW sources. This is offline training/evaluation infrastructure rather than a live Android task environment. No repository license file or dataset-license declaration was found in the inspected sources; neither is inferred from the underlying ms-swift project.

Related resources

documented by: FedMABench: Benchmarking Mobile Agents on Decentralized Heterogeneous User Data

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

editorial primary review
Retrieved 2026-10-17T11:18:17Z. Supports: name, kind, summary, environments, topics, scope, notes, relations, details.benchmark_protocol.
Content fingerprint
b49e2b419918078ef9c68f3c9becc6c13d8fe530c165a36bdd61e370746eeea8
editorial primary review
Retrieved 2026-10-17T11:18:17Z. Supports: identity.
Content fingerprint
3e1b2b643a263af129b3a4092a84efe47d3643f00509f7b9517da21a4aac0371
editorial primary review
Retrieved 2026-10-17T11:22:54Z. Supports: release_status, details.repository_revision, licenses.code.
Content fingerprint
4fb977a469cee3d5083da0b5bf2984fedc6b867146a90b20963985e08f9e9716
editorial primary review
Retrieved 2026-10-17T11:23:06Z. Supports: identity, release_status.
Content fingerprint
df5379f4a9b350dad0c3d374d2f9a1d6eacd974b6d54750e2a14ecec1faec6e5
editorial primary review
Retrieved 2026-10-17T11:23:06Z. Supports: summary, details.benchmark_protocol, licenses.data.
Content fingerprint
2874269995d8c9f5dabe266433ecf724bf54c27af599d899f3f0090b8ee37cdc

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.