Pulling activity and event data

Introduction

Obtaining event detail from your IBM Security Verify tenant is as simple as two API calls which allows your SIEM, Business Intelligence (BI), and analytics tools to consume all audit events from the platform.

The IBM Security Verify event API allows for:

1. Page sizing
2. Filtering on event types (e.g. management, authentication, sso, etc)
3. Resource filtering (e.g. user, token, app_consent)
4. Timespan filtering (i.e. searching after a time, or between times)

❗️

Handling lots of events

The events API will max out at 10,000 events in the response. The API provides a simple way to allow you to make subsequent calls with ease. It includes a search_after object which includes a time and id which represents the last event in the current call, which allows you to make a second call that will not overlap events.

Gathering information

The API client used by the application must have the following permissions in IBM Security Verify:

• Manage reports
• Read reports

The application must have acquired an Access Token using the Client Credentials flow

Variables

The following variables are needed for this guide:

Get all events

The simplest call to make is get all events from the platform without filters. Though in larger environments, you will likely get over 10,000 records which needs to be handled through pagination.

curl -X GET "https://${tenant_url}/v1.0/events" -H "Authorization: Bearer ${access_token}"

Expected response:

{
    "response": {
        "events": {
            "search_after": {
                "total_events": 54,
                "max_size_limit": "false",
                "time": "1559156150862",
                "id": "def9ea72-30ce-4589-800a-7306e306ea2e"
            },
            "events": [
                {
                    "geoip": {
                        "continent_name": "North America",
                        "country_iso_code": "US",
                        "country_name": "United States",
                        "location": {
                            "lon": "-97.822",
                            "lat": "37.751"
                        }
                    },
                    {...},
                    {...},
                    {...}
                }...

To filter on specific events on certain types, we'll use a filter query string.

The query string for this call will include event_type.

Each event type will need to be escaped with \" on each side of the value.

1. \"management\" corresponds with management events
2. \"authentication\" corresponds with authentication events into the CI platform.
3. \"sso\" corresponds with SSO events.
   etc...

curl -X GET "https://${tenant_url}/v1.0/events?event_type=\"sso\"" -H "Authorization: Bearer ${access_token}"
-H 'Authorization: Bearer {token}'

You may string multiple together to multiples event types together, separated by commas. For example: event_type=\"sso\",\"authentication\"

Handling pagination (max records)

If your call response includes total_events that exceed the total events count in the response, or "max_size_limit": "true", you can make a second call with a filter as to not overlap with the first call.

For example, if the first call we made was:

GET https://mytenant.ice.ibmcloud.com/v1.0/events with a response of:

{
    "response": {
        "events": {
            "search_after": {
                "total_events": 19540,
                "max_size_limit": "false",
                "time": "1559156150862",
                "id": "def9ea72-30ce-4589-800a-7306e306ea2e"
            },

We know that:

1. The response includes 10,000 records, and
2. We've hit the max size limit, and
3. The last event in the response has the time of 1559156150862 and id of def9ea72-30ce-4589-800a-7306e306ea2e.

Therefore, our next call will look like:

GET /v1.0/events?after=\"1559156150862\",\"def9ea72-30ce-4589-800a-7306e306ea2e\" which responds with:

{
    "response": {
        "events": {
            "search_after": {
            "total_events": 19450,
            "max_size_limit": "false",
            "time": "1559227423000",
            "id": "f253f26d-bb00-497b-a508-b3ea0e4f6d6f"
        }
      },

When max size limit is false, then we know we do not need to make an additional call. If you are also using page sizing though, and your size is less than 10,000 then it will always say max_size_limit false.

Using Parameters with Events API

The Events API is designed for customer data reporting and not to be part of real time flow. The average processing time for the events to show up may take up to 3000 ms, but it is not guaranteed. This article expands on advice and guidance on some of the Events API parameters:

1. Range type filtering: range_type - time or indexed_at -default is time, required
2. Use of after_id and after_time - must be used together for getting the next batch of events
3. Getting all event types: all_events - default is no, optional
4. Filtering on an event attribute using a filter_key and filter_value - defaults are: filter_key=data.performedby_type, filter_value="*", optional

Range

The Events API: /v1.0/events, by default gets the event for the last 24 hours. You can specify the time range. Sometimes, it seems that events are missing because there is a delay from the time a transaction completes on ISV to the point where the change is reported on the Events API. You can use the range_type, indexed_at instead of time. The default setting is time.

time: when the event was generated
indexed_at: when event was processed by the transform service.

Getting the next batch of events

The events API will max out at 10000 events in the response. To get the next batch of events, use both the time and id from the search_after object in the response from the current call for the subsequent calls.

after_id: The event ID of a previously returned event, after which to start searching. If the sort_order is ascending, then events that are generated or processed after this event are returned in increasing time. Note, the default sort order is descending. With this default the events are returned in decreasing time in the subsequent calls from the last event. If from and to values are included in the original request, keep them the same in the following requests to maintain the correct timeframe. To identify the event after which to start searching, after_id is the ID of the event after which to search and must be used in conjunction with after_time.
after_time: The event generation time (time) or event processing time by transform service (indexed_at) of a previously returned event, after which to start searching. If the sort_order is ascending, then the events generated or processed after this event are returned in increasing time for the subsequent calls. Note, the default sort order is descending. If from and to values are included in the original request, keep them the same in the following requests to maintain the correct timeframe To identify the event after which to start searching, after_time is the generation timestamp of the event after which to search and must be used in conjunction with after_id

curl -X GET 'https://TENANT_NAME/v1.0/events/?event_type=\"authentication\"&size=2&range_type=indexed_at' -H 'Authorization: Bearer XXX' -H 'Content-Type: application/json'

curl -X GET 'https://TENANT_NAME/v1.0/events/?event_type=\"authentication\"&size=2&range_type=indexed_at&after_id=d340e983-84e5-4216-8771-3c2d60e9d48c&after_time=1612292310689' -H 'Authorization: Bearer XXX' -H 'Content-Type: application/json'

All Events

Events API supports all_events filter to return certain types of event.
if all_events = no, which is the default. Events API will return only management, sso and authentication events.
if all_events = yes. Events API will return all event types: Currently all the event types that can be included are: management, sso, authentication, service, fulfillment, adaptive_risk, cert_campaign, access_request, account_sync, token or privacy_consent events.

Example with all_event=no

curl -X GET --header 'Accept: application/json' 'https://<tenantname>/v1.0/events?all_events=no&range_type=indexed_at' -H 'Authorization: Bearer <token>'

Example with all_event=yes

curl -X GET --header 'Accept: application/json' 'https://<tenantname>/v1.0/events?all_events=yes&range_type=indexed_at' -H 'Authorization: Bearer <token>'

Filtering on an event attribute

To filter on a specific event attribute we'll use filter_key and filter_value. The default value of the filter_key is data.performedby_type and filter_value="*" and is only applicable for events that have this attribute. You can use any attribute from the event to filter on. Note that the size limit of the event attribute is 32kb.

Here is an example of a sample event response :

{
    "response": {
        "events": {
            "search_after": {
                "total_events": 54,
                "max_size_limit": "false",
                "time": "1559156150862",
                "id": "def9ea72-30ce-4589-800a-7306e306ea2e"
            },
            "events": [
                {
                    "geoip": {
                        "continent_name": "North America",
                        "country_iso_code": "US",
                        "country_name": "United States",
                        "location": {
                            "lon": "-97.822",
                            "lat": "37.751"
                        }
                    },
                    {...},
                    {...},
                    {...}
                }...

Suppose we want to filter out the events from India. To get the desired result we need to put
filter_key = geoip.country_name and filter_value = \"India\"

curl -X GET --header 'Accept: application/json' 'https://<tenantname>/v1.0/events?all_events=no&filter_key=geoip.country_name&filter_value=%5C%22India%5C%22&range_type=time&x-forwarded-host=<tenant_id>'

Suppose we want to filter out events against a particular username ([email protected]). To get the result we need to put
filter_key = data.username and filter_value = \"[email protected]\"

curl -X GET --header 'Accept: application/json' 'https://<tenantname>/v1.0/events?all_events=no&filter_key=data.username&filter_value=%5C%22abc123%40ibm.com%5C%22&range_type=time&x-forwarded-host=<tenant_id>'