Help Center > Object Storage Service > Developer Guide > Managing Event Notification

Managing Event Notification

The notification function provided by OBS enables you to receive notifications when certain events occur in your bucket. To enable notifications, you must first add a notification configuration to identify the events that you want OBS to publish and the destinations to which you want OBS to send the event notifications.

Event Types Supported by OBS

OBS can publish events of the following types. You need to specify these event types in the notification configuration.

Table 1 Event types supported by OBS

Event Type

Description

s3:ObjectCreated:*

s3:ObjectCreated:Put

s3:ObjectCreated:Post

s3:ObjectCreated:Copy

s3:ObjectCreated:CompleteMultipartUpload

OBS can use APIs such as PUT, POST, and COPY to create objects and configure corresponding event types. You will receive a notification when an object is created using a specific API. In addition, you can use the s3:ObjectCreated:* event type to request all object creation notifications.

NOTE:

You will not receive event notifications from failed operations.

s3:ObjectRemoved:*

s3:ObjectRemoved:Delete

s3:ObjectRemoved:DeleteMarkerCreated

By using the ObjectRemoved event types, you can enable notification when an object or a batch of objects are removed from a bucket.

You can request notification when an object is deleted or a versioned object is permanently deleted by using the s3:ObjectRemoved:Delete event type. Alternatively, you can request notification when a delete marker is created for a versioned object by using s3:ObjectRemoved:DeleteMarkerCreated. You can also use s3:ObjectRemoved:* to request notification each time an object is deleted.

NOTE:

You will not receive event notifications from automatic deletions of lifecycle policies or from failed operations.

Destinations Supported by OBS

OBS can send event notification messages to the following destinations. You must grant the OBS permissions to send messages to these destinations. In addition, you need to specify the URN values of these destinations in the notification configuration.

  • Simple-Message-Notification (SMN) Topic
  • FunctionStage
  • FunctionGraph
    NOTE:
    1. SMN is a type of web service that a user can easily construct and maintain. SMN sends notifications from a cloud. SMN employs the publish-subscribe (pub-sub) message sending mode. Publishers send messages to the topics where they are authorized to publish messages. SMN takes the initiative to push messages to topic subscribers (such as email and SMS addresses).
    2. FunctionStage is an event-driven function hosting and computing service. With FunctionStage, you only need to compile code of service functions and configure the function running conditions. It eliminates the needs to configure or manage infrastructures (such as servers) and provides a scalable, maintenance-free, and reliable running environment for functions.
    3. FunctionGraph enables you to orchestrate service processes in a visualized manner, and coordinates a series of function components to control service processes. Each function component can execute a group of customized functions. With the easy workflow definition language of FunctionGraph, you can quickly construct the service process.

How to Enable Event Notifications

Enabling notifications is a bucket-level operation. OBS stores your event notification configuration to bucket sub-resources in the format of XML. By default, notification is not enabled for any event type. That is, the initial event notification configuration of each bucket is empty.

To enable notifications for events of specific types, you must add the corresponding XML configuration that identifies the event types you want OBS to publish and the destination where you want the notifications to be published, for example,

  • publishing event messages to an SMN topic — To set an SMN topic as the notification destination for specific event types, add the TopicConfiguration.
    <NotificationConfiguration>
      <TopicConfiguration>
        <Id>optional-id-string</Id>
        <Topic>topic-urn</Topic>
        <Event>event-type</Event>
        <Event>event-type</Event>
         ...
      </TopicConfiguration>
       ...
    </NotificationConfiguration>
  • Call a FunctionStage function and provide an event message as a parameter called by the function. To set the function as the notification target of a specific event type, add CloudFunctionConfiguration.
    <NotificationConfiguration> 
        <CloudFunctionConfiguration>
            <Id>ConfigurationId</Id>
            <Filter>
            ...
            </Filter>
            <CloudFunction>cloud-function-arn</CloudFunction>
            <Event>event-type</Event>
            ...
            </CloudFunctionConfiguration>
            ...
    </NotificationConfiguration>
  • Execute a FunctionGraph workflow and provide an event message as a parameter called by the function workflow. To set the function as the notification target of a specific event type, add FunctionGraphConfiguration.
    <NotificationConfiguration> 
        <FunctionGraphConfiguration>
            <Id>ConfigurationId</Id>
            <Filter>
            ...
            </Filter>
            <FunctionGraph>function-graph-arn</FunctionGraph>
            <Event>event-type</Event>
            ...
        </FunctionGraphConfiguration>
        ...
     </NotificationConfiguration>

To remove all notification configurations from a bucket, set the <NotificationConfiguration> element to null.

Using Object Key Names to Filter Event Notifications

You can set event notification by the prefix and suffix of an object key name. For example, you can set up a configuration so that a notification is published only when objects with a .jpg extension are added to a bucket.

OBS stores the notification configuration as XML. You can use the Filter element in the XML structure to define the rules for notifications to be filtered by the prefix and/or suffix of an object key name. Notification configurations that use the filter cannot define filtering rules with overlapping prefixes, overlapping suffixes, or prefix and suffix overlapping. The following are examples of notification configurations with object key name filtering:

  • Example of valid notification configuration with object key name filtering

    The following information contains a configuration identifying an SMN topic for OBS to publish events of the s3:ObjectCreated:Put type to. The events will be published each time an object that has an image prefix and a jpg suffix is PUT to a bucket.

    <NotificationConfiguration>
      <TopicConfiguration>
        <Id>01</Id>
        <Filter>
          <S3Key>
            <FilterRule>
              <Name>prefix</Name>
              <Value>image</Value>
            </FilterRule>
            <FilterRule>
              <Name>suffix</Name>
              <Value>jpg</Value>
            </FilterRule>
          </S3Key>
        </Filter>
        <Topic>urn:smn:southchina:11aa22bb:s3notification</Topic>
        <Event>s3:ObjectCreated:Put</Event>
      </TopicConfiguration>
    </NotificationConfiguration>

    The following notification configuration has multiple non-overlapping prefixes. The configuration defines the following: When objects that have a prefix of images are uploaded to buckets, event notifications will be published to topic-A; when objects that have a prefix of videos are uploaded to buckets, event notifications will be published to topic-B.

    <NotificationConfiguration>
      <TopicConfiguration>
        <Id>01</Id>
        <Filter>
          <S3Key>
            <FilterRule>
              <Name>prefix</Name>
              <Value>images</Value>
            </FilterRule>
          </S3Key>
        </Filter>
        <Topic>urn:smn:southchina:11aa22bb:topic-A</Topic>
        <Event>s3:ObjectCreated:Put</Event>
      </TopicConfiguration>
      <TopicConfiguration>
        <Id>02</Id>
        <Filter>
          <S3Key>
            <FilterRule>
              <Name>prefix</Name>
              <Value>videos</Value>
            </FilterRule>
          </S3Key>
        </Filter>
        <Topic>urn:smn:southchina:11aa22bb:topic-B</Topic>
        <Event>s3:ObjectCreated:Put</Event>
      </TopicConfiguration>
    </NotificationConfiguration>

    The following notification configuration has multiple non-overlapping suffixes. This configuration defines the following: All .jpg objects uploaded (PUT) to buckets are processed by function A, and all .png objects are processed by function B. The suffixes .png and .jpg are not overlapping even though they have the same last letter.

    <NotificationConfiguration> 
       <CloudFunctionConfiguration> 
         <Id>01</Id> 
         <Filter> 
           <S3Key> 
             <FilterRule> 
               <Name>suffix</Name> 
               <Value>.jpg</Value> 
             </FilterRule> 
           </S3Key> 
         </Filter> 
         <CloudFunction>urn:fss:southchina:11aa22bb:funct-A</CloudFunction> 
         <Event>s3:ObjectCreated:Put</Event> 
       </CloudFunctionConfiguration> 
       <CloudFunctionConfiguration> 
         <Id>02</Id> 
         <Filter> 
           <S3Key> 
             <FilterRule> 
               <Name>suffix</Name> 
               <Value>.png</Value> 
             </FilterRule> 
           </S3Key> 
         </Filter> 
         <Topic>urn:fss:southchina:11aa22bb:funct-B</Topic> 
         <Event>s3:ObjectCreated:Put</Event> 
       </CloudFunctionConfiguration> 
    </NotificationConfiguration>
  • Examples of notification configuration with invalid prefix/suffix overlapping

    In most cases, notification configurations that use the filter cannot define filtering rules with overlapping prefixes, overlapping suffixes, or overlapping combinations of prefixes and suffixes for the same event types. (You can have overlapping prefixes if the suffixes do not overlap.) You can use overlapping object key name filtering rules with different event types. For example, you can create a notification configuration that uses the image prefix for the ObjectCreated:Put event type and the imageprefix for the ObjectDeleted:* event type.

    Configuration information that does not contain the filter attribute matches all prefixes and suffixes by default. The following notification configuration is invalid because it contains overlapping prefixes. (The same thing would be true if suffix instead of prefix is used in this example.)

    <NotificationConfiguration>
      <TopicConfiguration>
        <Topic>urn:smn:southchina:11aa22bb:topic-A</Topic>
        <Event>s3:ObjectCreated:*</Event>
      </TopicConfiguration>
      <TopicConfiguration>
        <Filter>
          <S3Key>
            <FilterRule>
              <Name>prefix</Name>
              <Value>abc</Value>
            </FilterRule>
          </S3Key>
        </Filter>
        <Topic>urn:smn:southchina:11aa22bb:topic-B</Topic>
        <Event>s3:ObjectCreated:*</Event>
      </TopicConfiguration>
    </NotificationConfiguration>

    The following notification configuration is invalid because it contains overlapping suffixes. Two suffixes are considered overlapping if a given string can end with both suffixes. A string can end with jpg and pg. Therefore, the suffixes are overlapping. (The same is true for prefixes.)

    <NotificationConfiguration> 
       <FunctionGraphConfiguration> 
         <Filter> 
           <S3Key> 
             <FilterRule> 
               <Name>suffix</Name> 
               <Value>jpg</Value> 
             </FilterRule> 
           </S3Key> 
         </Filter> 
         <FunctionGraph>urn:fgs:southchina:11aa22bb:funct-work-flow-A</ FunctionGraph> 
         <Event>s3:ObjectCreated:*</Event> 
       </FunctionGraphConfiguration> 
       <FunctionGraphConfiguration> 
         <Filter> 
           <S3Key> 
             <FilterRule> 
               <Name>suffix</Name> 
               <Value>pg</Value> 
             </FilterRule> 
           </S3Key> 
         </Filter> 
         <FunctionGraph>urn:fgs:southchina:11aa22bb:funct-work-flow-B</ FunctionGraph>
         <Event>s3:ObjectCreated:Put</Event> 
       </FunctionGraphConfiguration> 
    </NotificationConfiguration>

Event Message Structure

A notification message sent by the OBS to publish an event is a JSON message with the following structure. Note the following:

  • If you want to trace the request by following up with the OBS support, the responseElements key value is useful. Both x-amz-request-id and x-amz-id-2 help OBS to trace the individual request. These values are the same as those returned by the OBS in response to your original request.
  • The S3Key provides information about the bucket and object involved in the event. Note that the object key name value is URL encoded. For example, red flower.jpg becomes red+flower.jpg.
  • The sequencer key provides a way to determine the sequence of events. Event notifications are not guaranteed to arrive in the sequence that the events occurred. However, notifications from events that create objects (PUTs) and delete objects contain a sequencer, which can be used to determine the sequence of events for a given object key. If you compare the hexadecimal sequencer strings from two event notifications on the same object key, you will find that the event notification with the greater sequencer value is the later event.
    NOTICE:
    1. The sequencer cannot be used to determine the sequence for events on different object keys.
    2. The value of sequencer is for reference only, and cannot be used in highly reliable systems
{  
   "Records":[  
      {  
         "eventVersion":"2.0",
   
         "awsRegion":"southchina",
         "eventTime":The time, in ISO-8601 format, for example, 1970-01-01T00:00:00.000Z, when S3 finished processing the request,
         "eventName":"event-type",
         "userIdentity":{  
            "principalId":"Customer-ID-of-the-user-who-caused-the-event"
         },
         "requestParameters":{  
            "sourceIPAddress":"ip-address-where-request-came-from"
         },
         "s3":{  
            "s3SchemaVersion":"1.0",
            "configurationId":"ID found in the bucket notification configuration",
            "bucket":{  
               "name":"bucket-name",
               "ownerIdentity":{  
                  "principalId":"Customer-ID-of-the-bucket-owner"
               },
               "arn":"bucket-ARN"
            },
            "object":{  
               "key":"object-key",
               "size":object-size,
               "eTag":"object eTag",
               "versionId":"object version if bucket is versioning-enabled, otherwise null",
               "sequencer": "a string representation of a hexadecimal value used to determine event sequence, 
                   only used with PUTs and DELETEs"            
            }
         }
      }
   ]
}  

Example message:

  • Test message — When you configure an event notification on a bucket, OBS sends the following test message:
    {  
       "Service":"OBS",
       "Event":"s3:TestEvent",
       "Time":"1970-01-01T00:00:00.000Z",
       "Bucket":"bucketname",
       "RequestId":"0002F4BCF6000001563B064B17B2094D",
       "HostId":"2Zf+b9AmzbaBgNQ+YE8XU2j87DZaBNxu4TaMiOCTqpmkC2SA9ouf8TpB2SY5j3i4"
    }
  • Example message when an object is created using the PUT request — The following is an example of a message sent by OBS to publish an s3:ObjectCreated:Put event:
    {  
       "Records":[  
          {  
             "eventVersion":"2.0",
            
             "awsRegion":"southchina",
             "eventTime":"1970-01-01T00:00:00.000Z",
             "eventName":"ObjectCreated:Put",
             "userIdentity":{  
                "principalId":"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
             },
             "requestParameters":{  
                "sourceIPAddress":"127.0.0.1"
             },
            
               
                
            
             "s3":{  
                "s3SchemaVersion":"1.0",
                "configurationId":"notificationConfigRule",
                "bucket":{  
                   "name":"bucket.notification",
                   "ownerIdentity":{  
                      "principalId":"4ca9f23fb9e54e4882ee22c6a1101163"
                   },
                   "arn":"arn:aws:s3:::bucket.notification"
                },
                "object":{  
                   "key":"notification.jpg",
                   "size":1024,
                   "eTag":"3f31f916d93271fe11a0815266e5be92",
                   "versionId":"0000015644343C1Bea5ec7bf799e61b67cb71e958eb78b53000655445346485a",
                   "sequencer":"015644343C28159329"
                }
             }
          }
       ]
    }
    NOTE:

    Due to Internet issues or constraints on e-mail sending, notifications may fail to send to HTTP or e-mail terminals.

Registration