DEBS 2026: Call for Challenges

Since 2011, the DEBS Grand Challenge has anchored the community’s benchmarking culture with yearly, data-backed problems. The Challenge has spanned multiple domains, including social games, manufacturing equipment, soccer monitoring, smart homes, taxi trips, social networks, RDF, maritime transportation with ML, LiDAR, NILM, AQI, market data, and HDD telemetry, while holding evaluation steady on correctness, throughput, and latency. Some challenges (e.g., the 2015 taxi trips) became de facto benchmarks, pushing stream processing beyond the DEBS limits.

This year, we propose an alternative format: a “Call for Challenges” that invites companies and researchers to contribute well-specified, evaluable problems backed by representative datasets and well-defined evaluation criteria.

Selected proposals will have the opportunity to become next year’s DEBS Grand Challenge, gaining visibility and attracting original solutions from experts in event and stream processing.

Why should a company participate? The pitch for your boss/CEO

• Strategic R&D leverage: turn a real backlog problem into a public benchmark; obtain diverse solution ideas without contracting multiple vendors.
• Technical hiring magnet: attract and identify top candidates through leaderboard results, artifact reviews, and finalist Q&A.
• Technology benchmarking: receive an independent, apples-to-apples comparison of relevant methods on your data characteristics (skew, bursts, late events, drift).
• Standards influence: shape evaluation metrics and harness features used by academia and industry in subsequent years.
• Brand amplification: selected challenges will be presented at DEBS and top proposals will be adopted as official DEBS Grand Challenge problems in the following year, potentially becoming long-term community benchmarks.

What is needed

• Problem statement: domain, queries/tasks, KPIs, constraints, success criteria.
• Dataset and its spec: schema, size, velocity, skew, nulls, anomalies,
• Operations: access modality (files, API, enclave), reference hardware profile, allowed dependencies.
• Evaluation criteria:
• Minimum: correctness tests, metric definitions.
• Good: ground truth to test the accuracy of the results.
• Ideal: baseline solution to compare against.
• Legal: data license/DUA, IP expectations, publicity terms, export restrictions.

Submissions

Participants are expected to submit a short paper (maximum 6 pages) describing the proposed challenge and the main technical difficulties it introduces https://cmt3.research.microsoft.com/DEBS2026. The paper should clearly describe the datasets, the evaluation criteria, and the access modalities, and motivate the relevance of the proposal to the DEBS community.

Review and selection criteria

Submitted papers will be reviewed by the DEBS Grand Challenge Program Committee. Each proposal will be evaluated based on the following criteria:

Originality and novelty of the proposed challenge, including its potential to stimulate new research directions and non-trivial solutions in event and stream processing.

Relevance and alignment with the topics of the DEBS conference, and with the Grand Challenge tradition.

Quality, representativeness, and availability of the dataset, including clarity of the data specification and feasibility of access for participants.

Soundness and clarity of the evaluation methodology, including well-defined metrics, correctness criteria, and comparison procedures.

Reviewers will assess both the scientific interest of the problem and its suitability as a benchmark-style challenge for the community.

Outcome of the Call for Challenges

This Call for Challenges directly feeds into the next edition of the DEBS Grand Challenge.

Among the accepted submissions, one or more top proposals will be selected and used as official DEBS Grand Challenge problems for the subsequent year.

Authors of the selected proposals will be invited to collaborate with the DEBS Grand Challenge organizers in finalizing the problem specification, datasets, and evaluation process, and may be involved in the assessment of submitted solutions.