What's new-Archive and replay events in Amazon EventBridge

The event-driven architecture uses events to share information between the components of one or more applications. The event will tell you what happened, including that you received an API request, that the file was uploaded to the storage platform, and that the database record has been updated. Business activities describe things related to your activities, such as the creation of a new customer account and successful payment.

You can use Amazon EventBridge to connect to your application events, integrated Software as a Service (SaaS) applications, and AWS services to connect applications. This is a server event bus that provides real-time data flow from the source and routes the data to the destination, such as AWS Lambda.

The event will tell you the fact that if you are interested in this information, you can share it with anyone. After you create a new customer account, you can use this information in the new service you add without changing the existing interface. For example, a new fraud detection system may focus on performing security checks to understand all new customer accounts created to assess the possibility of fraud.

In some cases, you may need to reprocess past events. As shown below, there are many use cases that are useful.

To simplify this, EventBridge can now archive and replay events.

Archiving and playback handles all events handled by EventBridge, including events from the AWS platform, events from SaaS integration, and your own custom events.

EventBridge がリプレイ用に別のクォータを保持しているため、リプレイ中は、現在のイベントのスループットには影響しません。リプレイの速度は、1 秒あたりに公開されるイベントの観点から、リージョン内の現在の PutEvents 制限と同じです。PutEvents サービスのクォータを増やすように頼むと、リプレイが速くなります。このように、通常の操作はリプレイの影響を受けません。ただし、リプレイを処理するダウンストリームサービスのパフォーマンスと制限をチェックして、追加のワークロードを処理できるようにする必要があります。

進行中のリプレイは停止できます。停止したリプレイを再開することはできませんが、以前に停止したリプレイから、開始時刻を lastReplayedEvent タイムスタンプに設定して新しいリプレイを作成できます。

Let's look at how archiving and playback actually work.

Create an event archive create an event bus for the application in the EventBridge console.

次に、単純なイベントマッチングパターンを使用してイベントバスのルールを作成し、customerCreated と等しい DetailType で受信したすべてのイベントを Lambda 関数に送信して、イベント内の顧客データをデータベースに格納します。

Next, select the event bus, and then choose Archive events from the actions menu.

Name the name and description archive, and double check that the source is your event bus. An unlimited retention period is selected here, but you can choose the number of days to retain. After that, the event is automatically deleted from the archive.

Select the next step. Optionally, you can filter events archived by pattern matching syntax similar to the pattern matching syntax used when creating rules. Here, we decided not to add a filter, but to archive all events from the source.

Now, using this simple Python code, put some events on the bus, as shown below.

import jsonimport boto3EVENT_BUS = 'my-event-bus'# Create EventBridge clientevents = boto3.client('events')eventDetail= {'customerId': '123','customerName': 'Danilo','customerData': 'More info...'}# Put an eventresponse = events.put_events(Entries=[{'EventBusName': EVENT_BUS,'Detail': json.dumps(eventDetail),'DetailType': 'customerCreated','Source': 'com.mycompany.myapp'}])print(response['Entries'])

Execute the previous code multiple times. After a few minutes, you will find that my public activities have been archived, as shown below.

After the event was replayed for some time, I found a code error in the Lambda function. If the address is longer than expected, the function will not store all the data (yes, will this happen?). Modify the code to ensure that the new event is handled correctly. You can also use EventBridge to replay old events to modify the results of handled events.

Because this application is idempotent, you can do this safely. That is, you can run it multiple times with the same input without changing the result. In this case, the database project will be overwritten with the correct customer data. We recommend that you always design idempotent interfaces to build well-resilient applications. By doing so, you can easily recover from these errors or safely ignore duplicate input.

Note that EventBridge does not provide a sequence guarantee, and the replay is multithreaded, so events may be distributed in a different order than the original order.

With the archive selected, click start New playback, and then enter a name and description for the replay.

Make sure the source of the replay is their file. Choose to replay only the rules that trigger the corrected Lambda function.

Define the start and end times of the playback (you can use local time or UTC), and then select start playback.

My file is not that big. After the replay starts, it only takes a few seconds to complete. If you look at your own database, you will find that the customer data is correct. This fixes a mistake!

Event archiving and playback are available in all commercial areas except Chinese mainland and Osaka. For more information, see the list of different services in the AWS area. This new feature can be used in the console, AWS command line interface (CLI), AWS SDK, and AWS CloudFormation.

Amazon EventBridge only pays the amount you use. For archiving and playback, you will be paid for the number of archived events, the storage used by the archive, and the number of messages to be replayed. For more information and examples of costs, see the rate table page of EventBridge.

For more information, see the documentation here. Please be sure to tell me how you can take advantage of this new feature.

-Danilo