Data Prepper 2.13 brings native OpenSearch data streams and Prometheus integration - OpenSearch

Data Prepper 2.13 brings native OpenSearch data streams and Prometheus integration

By [Krishna Kondaka](/content/author/krishna-kondaka/ "Posts by Krishna Kondaka"/index.html), [David Venable](/content/author/david-venable/ "Posts by David Venable"/index.html) December 3, 2025

The OpenSearch Data Prepper maintainers are happy to announce the release of Data Prepper 2.13. This release includes a number of improvements and new capabilities that make Data Prepper easier to use.

Prometheus sink

Data Prepper now supports Prometheus as a sink—initially, only Amazon Managed Service for Prometheus is supported as the external Prometheus sink. This enables you to export metric data processed within Data Prepper pipelines to the Prometheus ecosystem and allows Data Prepper to serve as a bridge between various metric sources (like OpenTelemetry, Logstash, or Amazon Simple Storage Service [Amazon S3]) and Prometheus-compatible monitoring systems.

A core aspect of the Prometheus sink is its handling of different metric types. The implementation ensures that Data Prepper’s internal metric representations are correctly mapped to Prometheus time series families:

In addition to mapping metrics, the sink handles attribute labeling and name sanitization, creating labels for all metric, resource, and scope attributes.

It can be easily configured for Amazon Managed Service for Prometheus as follows:

sink:
  - prometheus:
      url: <amp workspace remote-write api url>
      aws:
         region: <region>
         sts_role_arn: <role-arn>

OpenSearch data stream support

Data Prepper now supports OpenSearch data streams natively in the opensearch sink. With this change, Data Prepper will look up the index to determine whether it is a data stream. If so, it will configure the bulk writes to the sink so that they work directly with data streams.

Prior to this feature, Data Prepper pipeline authors would need to make manual adjustments to the sink configuration to write to data stream indexes. Now users can create a minimal sink configuration that will set up the sink correctly. Additionally, Data Prepper will automatically set the @timestamp field to the time received by Data Prepper if the pipeline does not already set this value.

For example, the configuration could be as simple as the following:

sink:
  - opensearch:
      hosts: [ "https://localhost:9200" ]
      index: my-log-index

Cross-Region s3 source

The s3 source is a popular Data Prepper feature for ingesting data from S3 buckets. This source can read from S3 buckets using Amazon Simple Queue Service (Amazon SQS) notifications or scan multiple S3 buckets. It is common for users to have S3 buckets in multiple AWS Regions that they want to read in a single pipeline. For example, some teams may want to get VPC flow logs from multiple Regions and consolidate them into a single OpenSearch cluster. Now Data Prepper users can read from multiple buckets in different Regions. And there is no need to create a custom configuration for this feature—Data Prepper will handle this for customers.

Other great changes

Getting started

Thanks to our contributors!

Thanks to the following community members who contributed to this release!