ArcSight CEF format

The Common Event Format (CEF) standard format, developed by ArcSight, enables vendors and their customers to quickly integrate their product information into ArcSight ESM.

CEF defines a syntax for log records comprised of a standard header and a variable extension, formatted as key-value pairs.

When syslog is used as a transport mechanism, CEF uses the following format, comprised of a syslog prefix, a header, and an extension:

The following example illustrates a general CEF message using syslog transport:

Using CEF Without Wyslog

Syslog applies a syslog prefix to each message, no matter what device it arrives from, which contains the date and hostname:

Jan 18 11:07:53 host CEF:Version\|…

However, if an event producer is unable to write syslog messages, it is still
possible to write the events to a file. In this case, begin the message with the format shown below, and omit the syslog prefix:

<!--

Sample Python functions for CEF creation

This section describes a set of sample Python functions for generating
CEF-formatted CP events.

There are three main functions in this package:

fetch_oauth_token()
query_events()
cef_generator()

Using the functions to demonstrate sample usage

Prerequisite: Python 3.5 or above

Follow these steps:

Download the Python code from
https://github.com/centrify/centrify-hparcsight-integration-sample/
Install pip packages in requirement.txt.
Provide the values for tenant, siem_username, and siem_password in config.ini.
Execute sample_usage.py to generate CEF-formatted CP events for one hour:

python3.5 sample_usage.py

The following example shows a CEF message for a Self-Service App Launch CIS
Event:

CEF:0|Centrify|Centrify_Cloud|1.0|Cloud.Saas.Application|Cloud.Saas.Application.SelfServiceAppLaunch|5|dhost=AAA0056 duser=cloudadmin@persistent.com01 msg=User cloudadmin@persistent.com01 launched Instagram from 103.6.32.100 shost=103.6.32.100 src=103.6.32.100 rt=1525844566655 deviceProcessName=centrify-syslog-writer dvchost=dinesh-VirtualBox dtz=Africa/Abidjan requestContext=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063 externalId=772a4a904e82da87.W00.0315.1aa20afe647f09c dpriv=WebRole destinationServiceName=CDS suid=c2c7bcc6-9560-44e0-8dff-5be221cd37ee cs1=Instagram cs1Label=applicationId cs2=Instagram cs2Label=applicationName cs3=Web cs3Label=applicationType cs4=103.6.32.100 cs4Label=clientIPAddress cs5=65f79bb1-4f91-4496-9991-d148da16cc3e cs5Label=internalSessionId cs6=0d10a24f4c57434198fb3ad4559cc48b cs6Label=azDeploymentId directoryServiceNameLocalized=Centrify Directory threadType=RestCall azRoleId=WebRole_IN_0 internalTrackingID=d3a0713b610146ca916155efca2be690 authMethod=UserPassword requestIsMobileDevice=False directoryServiceUuid=09B9A9B0-6CE8-465F-AB03-65766D33B05E requestDeviceOS=Windows level=Info

You can customize the usage or the APIs per your application needs.

CEF has a predefined set of keys.

CEF Mapping of CP Events

This section provides detailed information about how the CEF fields have been mapped from the CP event fields in the Python application described above.

CEF header

Header Field	CP Event Field
Version	‘0’
Device Vendor	‘Centrify’
Device Product	‘Centrify_Cloud’
Device Version	‘1.0’
Device Event Class ID	Variable — depends on the event. For example: ‘Cloud.Saas.Application’
Name	Variable — depends on the event. For example: Cloud.Saas.Application.SelfServiceAppLaunch’
Severity	Variable — depends on the Level field in event. For example: ‘5’ for Info, ‘10’ for Error.

CP ArcSight CEF extension

The CEF Extension contains a collection of key-value pairs. The keys are
predefined and are referred to as the ArcSight Extension Dictionary. (CEF Fields)

Common properties in CP events

This section lists the CEF field mapping of CP events, which are part of the CEF extension.

These properties are common to all events of Cloud Suite and Privilege Services.

ArcSight CEF Field	CP Event Field
The common properties are those listed below in bold.
Destination Host Name	Tenant
Destination User Name	NormalizedUser
Message	EventMessage
Source Host Name	RequestHostName
Source Address	FromIPAddress
Device Receipt Time	whenoccurred_epoch_ms (This is the event timestamp in UTC)
Device Process Name	‘centrify-syslog-writer’ (can be configured in cef_mapping.ini)
Device Host Name	Hostname of machine running the python app
Device Time Zone	‘Africa/Abidjan’ Note: This time zone is chosen mainly to set UTC offset to 0
The keys in the common properties section below are added in the CEF message only if no event-specific CEF mapping is specified for an event in the mapping configuration file, which is enclosed with the Sample Python application for CEF creation.
Device Custom String 1	AuthMethod
Device Custom String1 Label	'authMethod'
Device Custom String2	RequestIsMobileDevice
Device Custom String2 Label	'requestIsMobileDevice'
Device Custom String3	DirectoryServiceUuid
Device Custom String3 Label	'directoryServiceUuid'
Device Custom String4	RequestDeviceOS
Device Custom String4 Label	'requestDeviceOS'
Device Custom String5	Level
Device Custom String5 Label	Level
-->

Event-specific properties in CP

This section lists the event-specific properties mapped to ArcSight Fields. All events (whether they are listed below or not) will have the first nine common properties, identified in the table above, mapped in an ArcSight.CEF message.

Any CEF key appearing in event-specific mapping will override the CEF key mapping in the common properties section. For example, the
Cloud.Server.ManualAccount.SessionStart event, Destination host (Dhost), and Destination User(duser) will be ‘ComputerName’ and ‘AccountName’, which will overwrite the common properties mapped for dhost and duser.

EventType=Cloud.Core.MfaSummary

ArcSight CEF Field	CP Event Field
Reason	MfaReason
Outcome	MfaResult
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Device Custom String 1	MfaInitiator
Device Custom String1 Label	‘mfaInitiator’
Device Custom String2	FactorsLocalized
Device Custom String2 Label	‘factorsLocalized’
Device Custom String3	ProfileName
Device Custom String3 Label	‘profileName’
Device Custom String4	FailReason
Device Custom String4 Label	‘failReason’
Device Custom String5	MfaUnlock
Device Custom String5 Label	‘mfaUnlock’
Device Custom String6	ForgotPassword
Device Custom String6 Label	‘forgotPassword’
Device Custom Number1	Factorcount
Device Custom Number1 Label	‘factorCount’
Device Custom Number2	SecurityQuestionAnswerCount
Device Custom Number2 Label	‘securityQuestionAnswercount’

The remaining fields in an event that are not mapped to CEF keys will still be added in the CEF message with their CP-event field keys. These custom non-CEF keys will not be available for reporting in ArcSight, but they can viewed as part of the raw event message.

EventType=Cloud.Saas.Application.AppLaunch

ArcSight CEF Field	CP Event Field
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Suid	UserGuid
Device Custom String 1	ApplicationID
Device Custom String1 Label	‘applicationId’
Device Custom String2	ApplicationName
Device Custom String2 Label	‘applicationName’
Device Custom String3	ApplicationType
Device Custom String3 Label	‘applicationType’
Device Custom String4	TemplateName
Device Custom String4 Label	‘templateName’
Device Custom String5	InternalSessionId
Device Custom String5 Label	‘internalSessionId’
Device Custom String6	AzDeploymentId
Device Custom String6 Label	azDeploymentId

EventType=Cloud.Saas.Application.GatewayAppLaunch

EventType=Cloud.Saas.Application.SelfServiceAppLaunch

ArcSight CEF Field	CP Event Field
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Suid	UserGuid
Device Custom String 1	ApplicationID
Device Custom String1 Label	‘applicationId’
Device Custom String2	ApplicationName
Device Custom String2 Label	‘applicationName’
Device Custom String3	ApplicationType
Device Custom String3 Label	‘applicationType’
Device Custom String4	ClientIPAddress
Device Custom String4 Label	‘clientIPAddress’
Device Custom String5	InternalSessionId
Device Custom String5 Label	‘internalSessionId’
Device Custom String6	AzDeploymentId
Device Custom String6 Label	azDeploymentId

EventType=Cloud.Server.ManualAccount.SessionStart

EventType= Cloud.Server.LocalAccount.SessionStart

ArcSight CEF Field	CP Event Field
Src	FromIPAddress
Suser	NormalizedUser
Dhost	ComputerName
Duser	AccountName
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Suid	UserGuid
Device Custom String 1	UserType
Device Custom String1 Label	‘userType’
Device Custom String2	SessionType
Device Custom String2 Label	‘sessionType’
Device Custom String3	AuthorityName
Device Custom String3 Label	‘authorityName’
Device Custom String4	JumpType
Device Custom String4 Label	‘jumpType’
Device Custom String5	DirectoryServiceNameLocalized
Device Custom String5 Label	‘directoryServiceNameLocalized’
Device Custom String6	AuthoritySource
Device Custom String6 Label	‘authoritySource’

EventType=Cloud.Server.LocalAccount.PasswordExport

EventType= Cloud.Server.DomainAccount.PasswordExport

ArcSight CEF Field	CP Event Field
Src	FromIPAddress
Suser	NormalizedUser
Dhost	ComputerName
Duser	AccountName
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Suid	UserGuid
Device Custom String 1	UserType
Device Custom String1 Label	‘userType’
Device Custom String2	AuthorityID
Device Custom String2 Label	‘authorityID’
Device Custom String3	AuthorityName
Device Custom String3 Label	‘authorityName’
Device Custom String4	AzRoleId
Device Custom String4 Label	‘azRoleId’
Device Custom String5	DirectoryServiceNameLocalized
Device Custom String5 Label	‘directoryServiceNameLocalized’
Device Custom String6	CheckedOut
Device Custom String6 Label	‘checkedOut’
Device Custom Date1	WhenDueBack
Device Custom Date1 Label	‘whenDueBack’

EventType=Cloud.Core.Server.CpsTileLaunch

ArcSight CEF Field	CP Event Field
RequestContext	RequestUserAgent
ExternalId	ID
Dpriv	AzRoleName
DestinationServiceName	DirectoryServiceName
Suid	UserGuid
Device Custom String 1	UserType
Device Custom String1 Label	‘userType’
Device Custom String2	ApplicationType
Device Custom String2Label	‘applicationType’
Device Custom String3	ApplicationName
Device Custom String3Label	‘applicationName’
Device Custom String4	ApplicationID
Device Custom String4Label	‘applicationId’
Device Custom String5	DirectoryServiceNameLocalized
Device Custom String5Label	‘directoryServiceNameLocalized’
Device Custom String6	InternalTrackingID
Device Custom String6Label	‘internalTrackingID’

EventType=Cloud.Core.AdaptiveMfa.RiskAnalysis

Only Common properties.

Alternate approach for creating the Common Extension Format (CEF)

In case you are using the CP REST APIs directly in your application and generating your own Cloud Suite syslog messages in a generic non-CEF format having key=value pairs separated by a delimiter, then ArcSight SmartConnector will need to be installed and configured to collect these Cloud Suite syslog.

These logs will need to be parsed into CEF format by creating ArcSight
FlexConnector, to enable Cloud Suite events to be usable for SIEM in ArcSight. The only downside to using a FlexConnector is that ArcSight does not officially certify it.