SAP Data Services Agent Guide
SAP Data Services Agent Guide
10 Log Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
10.1 Log Retention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65
A Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
A.1 Collect Agent Diagnostic Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Using the Agent Diagnostic Tool User Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Running the Agent Configuration Tool via the Command Line. . . . . . . . . . . . . . . . . . . . . . . . . . . 74
A.2 Stopping the Internal Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
A.3 Manually Uninstalling the Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
The SAP Data Services Agent provides secure connectivity to on-premise sources in your landscape.
At design-time, the agent is used to provide metadata browsing functionality for on-premise sources to the
web based user interface. At run-time, the agent will take care of the secure data transfer from the on-premise
source to the targets in the cloud.
Note
While the SAP Data Services Agent is based on SAP Data Services technology, the two are not
interchangeable. If you want to connect to SAP Cloud Integration for data services, you must use the
SAP Data Services Agent.
We strongly recommend you use a version of the SAP Data Services Agent that is within four (4) releases of the
latest release of SAP Cloud Integration for data services.
Example:
With each new release of SAP Cloud Integration for data services, the minimum recommended version
increases by one version.
In addition, please be aware that in order to receive the most current features, functionality, and fixes, you must
use the latest release of the Agent. Updates and hot fixes to releases prior to the current release will not be
provided.
Caution
It is important that you do not update your Agent until the Server at your Data Center has been updated.
You can find your Data Center in your URL for SAP Cloud Integration for data services, which you will need
when referring to SAP Note 2324184 for the release schedule. This SAP Note also lists patch release
notes that detail the agent and server fixes for the respective releases and any known issues or limitations.
For additional release information, see the Product Availability Matrix (PAM).
SAP Cloud Integration for data services interacts with your local SAP landscape via the SAP Data Services
Agent and secure HTTPS and RFC connections.
Note
Even when your data flows from the cloud to your on-premise landscape, there is no need to open the
firewall to inbound traffic. The SAP Data Services Agent always initiates the request.
When used with SuccessFactors BizX, the SAP Cloud Integration for data services architecture is slightly
different:
In order to securely transfer data from your on-premise sources to the cloud, you must install and configure the
SAP Data Services Agent.
Before you begin the installation and configuration process, review the readiness checklist to ensure that you
have all the required information and understand each step that you need to perform.
Readiness checklist
1. Review the agent system requirements and ensure that your host system meets the minimum
requirements.
• Review Version Recommendations for SAP Data Services Agent [page 6] for important information
about the releases of SAP Data Services Agent you should use.
• For a detailed list of supported environments and hardware requirements, consult the Product
Availability Matrix (PAM). This information includes specific version and patch-level requirements for
web application servers, web browsers, databases, and operating systems.
Restriction
While the SAP Data Services Agent is based on SAP Data Services technology, the two are not
interchangeable. Additionally, for Windows host systems, the agent cannot be installed on a host
system where SAP Data Services or the Data Provisioning Agent for SAP Smart Data Integration
has already been installed. (This restriction does not apply to Linux host systems.)
• If you are installing the agent on a Linux system, ensure that your host system has the following
packages:
• X Window
• OpenGL libraries
• libgtk-2_0-0
• KornShell
• libncurses (if using SUSE 15.0 or higher)
Note
If you are installing the agent from release 2311 or later, libncurses6 is required due to an
SQL Anywhere dependency upgrade. For an agent from release 2309 or earlier, libncurses5 is
required.
If any packages are missing, the dependent libraries can be found as operating system patches.
Note
MS SQL support on Linux will use the pre-configured DataDirect ODBC driver that is bundled with
SAP Cloud Integration for data services.
Note
While you must run the SAP Data Services Agent installation program with administrative
privileges, the user account that will run the service does not require administrative privileges.
3. Collect administrator account information for SAP Cloud Integration for data services:
• User name and password for the SAP Cloud Integration for data services administrator account
4. Register an agent in the SAP Cloud Integration for data services web interface and download the
configuration file.
5. If you plan to use a proxy server, collect the necessary proxy information:
• Host name and port for your proxy server
• User name and password required by your proxy server (if required)
6. Ensure that you can access the URL that hosts the data center that communicates with the agent.
Note
You may need your IT administrator to add the datastore or data center URL to the allowlist to
obtain access.
7. Ensure that any necessary certificates are imported to your agent. See Importing Certificates [page
38] for more information.
3. If you plan to read from or write to flat files, compile a list of the directories that will be accessed.
Directories must be allowlisted in the SAP Data Services Agent before you can access them in SAP Cloud
Integration for data services.
4. If you plan to use web services (SOAP, RESTful or OData) that are secured with HTTPS, export the
necessary certificates from the server hosting the web service.
5. If you plan to connect to SAP Business Suite applications, prepare your SAP systems:
1. Install the required SAP function modules.
2. Create an SAP user with the required authorizations or assign the authorizations to an existing user.
The user you want to use to connect to the SAP Business Suite application requires the ZDSAUTH
authorization. ZDSDEV may also be used to further restrict access.
3. Configure an RFC connection, business extractors, and additional ABAP programs.
For more information about the required functions and user authorizations, see “Configuring SAP Business
Suite connectivity”.
6. If you plan to connect to a database, ensure that the correct connectivity drivers are installed on the host
system for your Data Services agent. Refer to the Product Availability Matrix (PAM) for middleware version
information. In all cases, the 64-bit version of the driver is required.
7. If you plan to run on Microsoft Windows, ensure that Microsoft Visual C++ 2019 is installed.
8. Install the SAP Data Services Agent.
After completing the installation and configuration process, log in to the SAP Cloud Integration for data
services web interface and see the Get Started tab for information about configuring projects and tasks.
Related Information
Depending on your requirements, you can use one or multiple agents to connect to SAP Cloud Integration for
data services.
You might choose to use multiple agents for any of the following reasons:
• Large data load volumes - divide the load between multiple agents
• Fail-over support - if one agent host system is down or unreachable, your tasks will still run
• Separate agents for test and production tasks
Restriction
Windows host systems can support only one installed agent. Linux host systems can support multiple
agents, but each agent must be run using a different operating system user.
When you use multiple agents, your datastores, projects, and other objects are not duplicated within SAP
Cloud Integration for data services. Instead, you select the agent or agent group to use at run-time when you
execute or schedule a task.
You can switch between agents freely as long as each agent is able to connect to the on-premise sources
required in your task. For agents that use flat-file sources, each agent needs access to its own copy of the files,
or you can use a network share to make them accessible to all agents.
Tip
When you edit a datastore connection, the agent that you choose is used only for metadata browsing. The
agent specified in the datastore is not used when you execute a task at run-time.
A one-to-one correlation exists between the agent configuration file that you download and install and the
machine that you install it on. You cannot reuse an agent configuration file on multiple machines or on
multiple users on Linux. If you are moving the agent from one machine to another machine, for example,
you must first delete the agent entry for the old computer before installing the agent on the new computer.
Do this within SAP Cloud Integration for data services, at the Agents tab.
For installations on Linux host systems, it's recommended that you use the following user resource limits. You
can display these settings by running the ulimit -a command.
lockedmem (kilobytes) 4
processes 7168
The SAP Data Services Agent installation program is distributed in a self-extracting executable.
• Windows: Run DataServices-Agent-Installer.exe. You must run the installation program from
a user account that has administrative privileges. On Windows platforms that have User Account
Control (UAC) enabled, you can also choose the Run as administrator option.
• Linux: Run DataServices-Agent-Installer.bin. Do not install as root.
After the package has been extracted, the installation program starts automatically.
2. Specify the path where you want to install the software.
• On Windows platforms, the default installation path is C:\Program
Files\SAP\DataServicesAgent.
• On Linux platforms, the default installation path is $HOME/DataServicesAgent.
3. On Windows, specify the path where the agent should store log files and settings. To prevent issues
when upgrading or applying patches, this path must be different than the installation path entered in the
previous step.
The logs and settings path is referenced by the <DS_COMMON_DIR> environment variable. The default path
is C:\ProgramData\SAP\DataServicesAgent.
4. Specify the user name and password for the local user account that will be used to run the job service.
Note
For domain user accounts, specify the user name using the format <DOMAIN>\<username>. For local
accounts, only the user name is required.
5. If you do not want to use the default ports, check Specify port numbers used by installation.
Specify new port numbers as required.
Note
If the installation program detects that the default ports are already in use, this option will be checked
automatically.
6. Click Install.
The installation progress displays. During the installation process, the installation program creates a log file
as follows:
• On Windows platforms, %DS_COMMON_DIR%\log\Install_<timestamp>.log
• On Linux platforms, <install_dir>\log\Install_<timestamp>.log
After the installation process is complete, you can choose to configure the Agent immediately or at a later time.
Use a WebSocket RFC-enabled connection to connect SAP Cloud Integration for data services and SAP
Integrated Business Planning.
There are two types of authentication for WebSocket RFC communication to an IBP datastore:
Related Information
Related Information
A personal security environment (PSE) is required to establish a WebSocket RFC connection. Use the X.509
certificate to authenticate via certificate to WebSocket RFC connections.
Prerequisites:
Note
To provide a high degree of security, the certificate in P12 format should be generated by the
connection owner using any certificate generator or generated and signed by a third-party certificate
authority (CA) listed in SAP Note 2871840 . The private key of this certificate should be kept in a
secure location and maintained by the connection owner.
A PSE is a secure container that stores the public-key information of a user or component. Creating a PSE
is mandatory to successfully authenticate and connect to a WebSocket RFC connection. The location of this
secure PSE container is defined by the agent during installation.
Note
• Personal Security Environment (PSE) files are loaded and generated from the SECUDIR variable and
they should not be moved or renamed. If moved, the PSE file will not work.
• The PSE file must be generated in the same machine as the agent. Again, it should not be shared,
moved, or renamed. If tasks are run against an agent group, generate a PSE file for each agent machine
within that group.
• If you alter an existing PSE file, functionality such as testing connections, browsing, and importing
require you to restart the agent to establish a WebSocket RFC connection. Job execution and job
run-time do not require you to restart the agent.
Caution
When a certificate used for a PSE expires, you must recreate the PSE from a new certificate.
Caution
It is important that you do this as an Administrator. Creating the file as a non-Administrator user
will produce an unusable file.
• Terminal on Linux
2. Go to the <agent_installation_folder>/bin directory.
3. Create a PSE file using information from the P12:
• If your SAP Integrated Business Planning datastore has different configurations for Sandbox and
Production, the PSE files should have different names between the two environments.
• You should use the same PSE file name within the same agents in the same group, so a task that is
sent to the agents in the same group will refer to the same PSE file name defined in the datastore.
For example:
Note
If switching from Basic Authentication to X.509, the PSE file is overridden and a new password
needs to be created.
4. Grant user access to the PSE file using the following command:
• (optional) Extract the public client certificate from the PSE file.
• Open the <public_client_certificate.crt> and remove the “BEGIN CERTIFICATE” and “END
CERTIFICATE” tags before uploading it to IBP because the IBP system does not accept BEGIN and
END tags in the certificate.
• Ensure that the client certificate is signed by one of the certificate authorities listed in 2871840 .
• Create/edit a communication arrangement and make sure the certificate is uploaded in the
communication user. For more information, see the "Using Basic Authentication" section of Defining
the Communication Arrangement.
6. Add the certificate to PSE. See Add the Server Certificate to the PSE [page 18].
Task overview: Connecting to SAP IBP Using Certificate-Based Authentication [page 15]
Next task: Add the Server Certificate to the PSE [page 18]
A server certificate is required to encrypt information and provide identity assurance in your PSE.
Caution
It is important that you do this as an Administrator. Creating the file as a non-Administrator user
will produce an unusable file.
• Terminal on Linux
3. Navigate to the SECUDIR folder.
4. Go to <agent_installation_folder>\bin.
5. Run sapgenpse to add the certificate to the PSE file:
To complete the WebSocket RFC setup, define the communication arrangements in the SAP Integrated
Business Planning UI. For more information, see “Defining the Communication Arrangement” in the SAP Cloud
Integration Guide available on the SAP Help Portal.
Task overview: Connecting to SAP IBP Using Certificate-Based Authentication [page 15]
Previous task: Create a Personal Security Environment with a PSE File [page 16]
Authentication by password requires a username, a password, and a Personal Security Environment (PSE) file.
Related Information
Prerequisite: Install or upgrade your agent to Patch 38 or higher.A personal security environment (PSE) is
required to establish a WebSocket RFC connection.
A personal security environment (PSE) is required to establish a WebSocket RFCA PSE is a secure container
where the public-key information of a user or component is stored. Creating a PSE is mandatory to
successfully authenticate and connect to a WebSocket RFC connection. The location of this secure PSE
container is defined by the agent during installation.
Note
• Personal Security Environment (PSE) files are loaded and generated from the SECUDIR variable and
they should not be moved or renamed. If moved, the PSE file will not work.
• The PSE file must be generated in the same machine as the agent. It cannot be shared, moved, or
renamed. If tasks are run against an agent group, generate a PSE file for each agent machine within
that group.
• If you alter an existing PSE file, functionality such as testing connections, browsing, and importing
require you to restart the agent to establish a WebSocket RFC connection. Job execution and job
runtime do not require you to restart the agent.
Caution
It is important that you do this as an Administrator. Creating the file as a non-Administrator user
will produce an unusable file.
• Terminal on Linux
2. Go to the <agent_installation_folder>/bin directory.
3. Run sapgenpse to generate the PSE file.
Note
Parent topic: Connecting to SAP IBP Using Password Authentication [page 18]
Next task: Add the Server Certificate to the PSE [page 20]
Related Information
A server certificate is required to encrypt information and provide identity assurance in your PSE.
Caution
It is important that you do this as an Administrator. Creating the file as a non-Administrator user
will produce an unusable file.
• Terminal on Linux
3. Navigate to the SECUDIR folder.
4. Go to <agent_installation_folder>\bin.
5. Run sapgenpse to add the certificate to the PSE file:
Task overview: Connecting to SAP IBP Using Password Authentication [page 18]
Previous: Create a Personal Security Environment with a PSE File [page 19]
• When you filter IBP data using an IN or NOT IN operator, the filtering occurs on the IBP side. Filtered results
are then provided to SAP Cloud Integration for data services, which increases performance efficiency. Prior
to patch 41, data was read from IBP and the filtering was done in memory on the Agent.
• For recommendations to help you set up data extraction via data flows using calculation scenarios, refer to
Best Practices for Extracting Data from SAP IBP. You can ignore the item in the section "Attribute-based
Filter" that suggests not using IN and NOT IN operators, as this is supported.
• If a job is reading or loading data and a connection attempt fails, the system will reattempt to connect
after the number of milliseconds set in the parameter Interval between Retries (ms) until it reaches the
value defined in the parameter Number of Connection Retries. If the reattempt fails, timestamped entries
will appear in the trace log. For more information about these parameters, see SAP Integrated Business
Planning via WebSocket RFC.
Related Information
Log Files
To use the SAP Data Services Agent to securely transfer your on-premise data with SAP Cloud Integration for
data services, you must configure your instance of the agent.
Prerequisite: Create the agent as described in Create an Agent before configuring it.
1. Register the agent in the SAP Cloud Integration for data services web interface.
2. Download the agent configuration file.
3. Configure the secure agent connection.
During initial configuration, or at a later time, you may need to change the software's configuration to meet
your requirements.
• Change the hostname of the SAP Cloud Integration for data services server.
• Add or remove directories that may be accessed by the agent.
• Change an adapter configuration.
• Uninstall the agent from the host system.
• For a BW target, you may want to set the parameter EmbeddedRFCShutdownTimeout to a short time such
as 60000 milliseconds (one minute) or even 5000 milliseconds (five seconds) in situations where multiple
jobs might start at the same time and they use the same RFC destination/PROGRAM ID (Registered Server
Program). If the protocol sends both requests to the same engine, this shorter parameter setting avoids
other engines from waiting the default of 10 minutes before timing out. For more information, see SAP
Note 3063345 .
Related Information
Before you can configure a local SAP Data Services Agent instance, you must register the agent in the SAP
Cloud Integration for data services interface.
After registering the agent, you can choose to download the configuration file immediately. If you plan to
configure the SAP Data Services Agent at a later time, you can download the configuration file later by
returning to the Agents section.
Related Information
Agent groups are collections of agents (typically in the same location) that are logically grouped to enable
high-availability solutions for your production tasks.
When you assign tasks to an agent group instead of an individual agent, SAP Cloud Integration for data services
can assign the task to any available agent in the group. You do not have to worry about whether a specific agent
is available or not. Administrators can create and configure agent groups in the Agents area of the SAP Cloud
Integration for data services web UI.
Note
Agents created before version 1.0.6 will be automatically assigned to a default agent group, which is named
after the organization.
When you configure the secure connection for an SAP Data Services Agent instance, you need to provide a
configuration file from SAP Cloud Integration for data services.
Related Information
After installing the SAP Data Services Agent, you must configure the secure connection before the agent can
be used with SAP Cloud Integration for data services.
Before you begin, register the agent in the SAP Cloud Integration for data services web interface and download
the configuration file.
1. If you did not choose to start configuration immediately after installation, start the SAP Data Services
Agent configuration program.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
Note
When you change the agent configuration, the SAP Data Services Agent service must be restarted
for the changes to take effect. You can choose to automatically restart the service when closing the
configuration program, or to manually restart the service at a later time.
Related Information
If you need to change the username and password used by the SAP Data Services Agent or your proxy
information has changed, you can update the agent configuration.
To reconfigure the agent, run the SAP Data Services Agent configuration program.
If you want to change the registration of the agent in SAP Cloud Integration for data services, you must
uninstall and reinstall the SAP Data Services Agent on the host system.
If your SOAP Web Services endpoints require client authentication, additional setup is necessary to enable this
authentication.
Restriction
This topic applies only if you are using Data Services Agent version 1.0.11 patch 34 or later.
Configure client authentication as described below, on both the server side and on the agent side.
Prerequisite: Before you perform the following steps, you must generate the keystore file (*.jks) and place it
on your agent machine to verify the client.
1. In SAP Cloud Integration for data services, under the Datastore tab, create a new SOAP Web Services
datastore or select a SOAP Web Services datastore to edit.
2. For the Keystore Path, enter the full path file name of the keystore at the agent. For example,
C:\FolderName\KeystoreFileName, or in Linux, /FolderName/KeystoreFileName. Refer to the
agent location that you set previously.
3. For the Keystore Password, enter the password for the keystore.
4. Click OK or Save to save the new or updated datastore.
For details about these and other SOAP Web Services datastore options, see SOAP Web Service in the HELP
CENTER.
Note
If you want a separate setting for each WS datastore, you can duplicate webservice-c folder and have
the datastore configuration point to that directory.
<!--
<parameter name="SERVER_CERT">/path/to/ca/certificate</parameter>
<parameter name="KEY_FILE">/path/to/client/certificate/chain/file</
parameter>
<parameter name="SSL_PASSPHRASE">passphrase</parameter>
-->
4. Update these values to refer to the server certificate, key file (pem file), and SSL passphrase.
For more information, see http://people.apache.org/~dumindu/docs/HowToConfigureSSL.html .
5. Restart the agent.
To read from and write to flat files in SAP Cloud Integration for data services, you must authorize the SAP Data
Services Agent to access directories on the host system.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure Directories.
The list of directories that the SAP Data Services Agent may access is displayed.
• To add a new directory to the list, click Add and browse to the directory that you want to add.
Note
Each directory must be explicitly declared. For example if you have a C:\Data directory with
C:\Data\2017 and C:\Data\2018 subdirectories that contain your data, you must add two
separate entries: one for C:\Data\2017 and another for C:\Data\2018
Restriction
The software does not support mapped drives. To add a network location, you must specify the
path using UNC notation. For example, \\<servername>\<sharename>.
• To remove a directory from the list, select the directory and click Remove.
4. If you are done configuring the SAP Data Services Agent, click Exit to close the configuration program.
Note
When you change the directory configuration, the SAP Data Services Agent service must be restarted
for the changes to take effect. You can choose to automatically restart the service when closing the
configuration program, or to manually restart the service at a later time.
To configure ODBC data sources in Linux, use the SAP Data Services Connection Manager.
SAP Cloud Integration for data services supports several ODBC data sources natively with DSN connections.
Ensure that SAP Cloud Integration for data services supports your ODBC data source. For more information,
see the Product Availability Matrix .
Ensure that you have the correct privileges to change the configuration files mentioned in these steps.
Install the GTK+2 library to make a graphical user interface for Connection Manager. The GTK+2 is a free
multi-platform toolkit that creates user interfaces. The installation is at https://www.gtk.org/ .
$ cd $LINK_DIR/bin/
$ ./DSConnectionManager.sh
Note
The Connection Manager does not save this information for further use.
To connect to web services (SOAP, RESTful or OData) that are secured with HTTPS, add your custom
certificates to the trusted certificates directory on the server hosting your Data Services agent.
Note
The manual process described here can be done automatically using the Import Certificates dialog in the
Data Services Agent Configuration tool.
1. Obtain a signed certificate from the server where the web service is hosted.
Export the certificate from the tools or settings of your web browser. The certificate must be saved with the
file extension .cer and start with -----BEGIN CERTIFICATE-----.
For Restful and SOAP web services, export the certificate in base-64 encoded X.509 (.cer) file format. For
OData, export the certificate in either base-64 encoded X.509 (.cer) or DER encoded binary X.509 (.cer)
file format.
2. Save your .cer file in the trusted_certs directory.
Running this command regenerates the keystore based on all certificates located in the trusted certificates
directory.
Note
You must run the command from a user account that has administrative privileges. On Windows
platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
4. Check if the file dsod.pem exists in the directory <DS_COMMON_DIR>\conf and then do one of the
following:
The Agent will scan the trusted_certs directory for all .cer files and add your .cer to the list of trusted
certificates.
Note
You must use a user account that has administrative privileges. On Windows platforms that have User
Account Control (UAC) enabled, you can also choose the Run as administrator option.
Related Information
To configure SSL support for SOAP web services, you must enable the SERVER_CERT parameter.
Ensure you have imported a signed certificate from the server where the web service is hosted. If needed, you
can import the certificate by one of the following methods: Importing Certificates [page 38] or Connecting to
Secure Web Services by Manually Adding Certificates [page 29]:
<!--<parameter name="SERVER_CERT">/path/to/ca/certificate</parameter>
-->
3. Remove the comment tags (<!-- -->) around the SERVER_CERT element.
4. In the SERVER_CERT parameter, enter the full path (including the certificate file name) to the CA certificate
stored in the trusted_certs directory.
<parameter name="SERVER_CERT"><LINK_DIR>\ssl\trusted_certs\<file_name.crt></
parameter>
For example:
To read from and write to a SuccessFactors instance, you must configure the SuccessFactors adapter in the
SAP Data Services Agent.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure Adapters.
The adapter configuration page is displayed.
3. Configure the SuccessFactors adapter as required for your instance.
Option Description
Adapter Retry The number of times the agent should attempt to start the adapter.
Count
Adapter Retry In- The amount of time the agent should wait between attempts to start the adapter, in milliseconds.
terval
Additional Java Additional options to use when starting the adapter instance. The default information for this
Launcher Op- parameter is -Xms64m –Xmx256m.
tions • The proxy can be disabled by removing the default proxy line from this field.
• To add a proxy server, append the proxy server parameters -Dhttp.proxyHost=<Your
proxy server name> and -Dhttp.proxyPort=<Your proxy port number>.
For example:
-Xms64m –Xmx256m -Dhttp.proxyHost=myproxy -Dhttp.proxyPort=8080
If you need to pass a username and password to your proxy as well, then also append:
-Dhttp.proxyUser=<Your proxy user name> -Dhttp.proxyPassword=<Your
proxy password>
Click Apply.
• To support a client authentication certificate, from the Agent Configuration Tool, copy the line
of generated output from the Configure Client Authentication tab that provides the keystore
and password. For example, Djavax.net.ssl.keyStore="C:\Program
Files\SAP\DataServicesAgent\ssl\client_certs\<keystoreName>.jks
" -Djavax.net.ssl.keyStorePassword=<*****>
Note
When you change the agent configuration, the SAP Data Services Agent service must be restarted
for the changes to take effect. You can choose to automatically restart the service when closing the
configuration program, or to manually restart the service at a later time.
To read from and write to an OData instance, you must configure the OData adapter in the SAP Data Services
Agent.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure Adapters.
The adapter configuration page is displayed.
3. Configure the OData adapter as required for your instance.
Option Description
Adapter Retry The number of times the agent should attempt to start the adapter.
Count
Adapter Retry In- The amount of time the agent should wait between attempts to start the adapter, in millisec-
terval onds.
Additional Java Additional options to use when starting the adapter instance. The default information for this
Launcher Options parameter is -Xms64m –Xmx256m.
• The proxy can be disabled by removing the default proxy line from this field.
• To add a proxy server, follow the instructions in Adding a Proxy Server [page 33]. Be sure
to click Apply after adding your proxy server parameters.
• To support a client authentication certificate, from the Agent Configuration Tool, copy the
line of generated output from the Configure Client Authentication tab that provides the key-
store and password. For example, -Djavax.net.ssl.keyStore="C:\Program
Files\SAP\DataServicesAgent\ssl\client_certs\<keystoreName>.j
ks" -Djavax.net.ssl.keyStorePassword=<*****>
Note
When you change the agent configuration, the SAP Data Services Agent service must be restarted
for the changes to take effect. You can choose to automatically restart the service when closing the
configuration program, or to manually restart the service at a later time.
Related Information
Append parameters for your proxy server when configuring an OData adapter.
When configuring the OData adapter as required for your instance, you can set Java Launcher options to use a
proxy server when the adapter instance starts.
Note
Be sure you know ahead of time whether your server's URL begins with http or https, as this is important
for configuring your OData adapter correctly.
You must generate a Java keystore output in order to authenticate a client certificate to be sent to the server.
A client certificate is sent from the client to the server at the start of a session and is used by the server to
authenticate the client. Follow these steps to generate and import the Java keystore that is used to verify the
client.
The Java keystore file will be generated under %LINK_DIR%\ssl\client_certs. It will also copy
the intermediate and root certificate that you downloaded from the chain certificate to %LINK_DIR%
\ssl\trusted_certs and import the certificates into the trustStore. You can now use the generated
output in the Configure Adapter tab of the Agent Configuration Tool to authenticate a client certificate.
Related Information
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Set up Agent in the menu on the left-hand side.
3. Enter your SAP Cloud Integration for data services administrator user name and password.
4. Download the latest Agent Configuration File from the new domain using the SAP Cloud Integration for data
services web UI. See Downloading the Agent Configuration File [page 24] for more information.
5. Navigate back to the SAP Data Services Agent configuration program and browse to the latest Agent
Configuration File you downloaded.
Note
If you are migrating to a new tenant, you will need to re-download the latest Agent Configuration File
and replace the Agent Configuration File in the configuration program.
Note
When you change the server host name, the SAP Data Services Agent service must be restarted
for the changes to take effect. You can choose to automatically restart the service when closing the
configuration program, or to manually restart the service at a later time.
Note
If you are updating the version of the agent on Windows or Linux platforms, ensure that you have done the
following prior to upgrading:
If you are using Linux, you can also automatically back up necessary files by stopping all processes using /
dsod_stop.sh and copying files to the backup using cp -rf.
To update the version of the agent installed on your host system, run the standard SAP Data Services Agent
installation program. When the installation program detects that an older version of the agent is already
installed on the host system, it automatically updates the existing installation instead of performing a new
installation.
Related Information
If you need to remove the SAP Data Services Agent from the host system, you can use a script to uninstall the
agent.
1. Close any open files, windows, or command prompts in the %LINK_DIR% or %DS_COMMON_DIR% folders.
If you don't close open files in these locations, the uninstallation script may be unable to remove all agent
files, and manual cleanup may be required.
2. Start the uninstallation process.
Note
You can also start the uninstallation process from Programs and Features in the Windows Control
Panel. Select the SAP Data Services Agent and click Uninstall.
Note
You must run the uninstallation script from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the uninstallation script is located in the directory where the SAP Data Services Agent was
installed on the host system.
The uninstallation script stops and removes the SAP Data Services Agent service, and removes all SAP Data
Services Agent files from the host system.
After uninstallation, uninstall.bat or uninstall.sh and uninstall.log will be left in the %LINK_DIR%
folder. If you want to remove all traces of the agent, you can manually remove these files after the uninstallation
script has finished.
Related Information
You may need to import new or updated certificates for secure communication between the Data Services
Agent and other servers such as those hosting web services or OData.
The Data Services Agent configuration tool eliminates the manual steps associated with updating the Data
Services Agent keystore.
1. If the SAP Data Services Agent configuration program is not already open, open it.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Import Certificates.
3. Specify the certificates you want to import using one of the following methods:
Select a certificate file SAP Cloud Integration for data Browse to the location of the updated
services server or new certificate.
Note
This is unusual. When needed,
updated certificates are included
in support package or patch re-
leases.
Proxy server
4. Click Import.
After the import is complete, the SAP Data Services Agent service automatically restarts.
If you will create a SOAP Web Service datastore that connects to a SOAP-based web service that uses SSL,
after importing the certificate you must place the keystore (*.jks) on your agent machine to verify the client.
If you want to use SAP Cloud Integration for data services to connect to your SAP Business Suite applications,
you must configure user authorizations and functions on the SAP application.
Related Information
The SAP Data Services Agent functions have a naming convention that includes a prefix.
The prefix of /SAPDS/ or /BODS/ is included with the corresponding SAP function names. The prefix depends
on the version of the SAP NetWeaver in use.
To extract data from an SAP Business Suite system, ensure that you run an SAP NetWeaver support package
that includes the required function modules.
Related Information
The SAP Data Services Agent functions are intended for use in either a development or production
environment.
Depending on the SAP NetWeaver version, the namespace for the Data Services Agent is /SAPDS/ or /
BODS/. For example, the fully qualified name of the AUTH_IMPORT function is either /SAPDS/AUTH_IMPORT
or /BODS/AUTH_IMPORT.
Development-only functions
• AUTH_IMPORT
• EXTRACTOR_IMPORT
• FUNCTION_GET
• IDOC_IMPORT
• RFC_ABAP_INSTALL_RUN
• TABLE_IMPORT
• TREE_IMPORT
• TREE_IMPORT40
• UPLOAD
Production functions
• ABAP_RUN
• BW_QUERY
• COLUMN_SEARCH
• DATA_PROFILE
• EXTRACTOR_NAVIGATE
• EXTRACTOR_SEARCH
• FILE_ROWCOUNT
• GET_VERSION
• IDOC_SEARCH
• JOB_LOG
• JOB_RUN
• JOB_STATUS
• MODEL_NAVIGATE
To access and integrate SAP Business Suite data, ensure that you have specific authorizations that support
SAP Data Services Agent operations.
Determine the required authorizations based on factors that include the following dependencies:
• Mode of transportation
• ABAP mode
• Source system version
As part of your planning process, determine your required authorizations and then request that they be
included in the profile associated with your SAP user.
Tip
For improved security, avoid using wildcards, generic, or blank values for authorization fields, especially in a
production environment. Instead use specific values that are appropriate to your business applications.
The following table helps you determine the required authorizations based on your specific needs.
Table 1:
Execute remote functions on an SAP server S_RFC: Authorization Check for RFC Access [page 50]
Access table data in an SAP system S_TABU_DIS: Table Maintenance [page 56]
• Access specific transactions S_TCODE: Authorization Check for Transaction Start [page
Access ERP hierarchies G_800S_GSE: Special Purpose Ledger Sets [page 46]
Stream using RFC S_RFC_ADM: Administration for RFC Destination [page 51]
Check DataSource access privileges S_RO_OSOA: SAP DataSource Authorizations [page 51]
Access the SAP Data Services Agent functions S_SDSAUTH: SBOP Data Services - General Authorization
[page 54]
SAP Data Services Agent-specific equivalent of the SAP S_SDSDEV: SBOP Data Services Authorization Object for
S_DEVELOP authorization object Development [page 54]
Define whether the SAP system should be treated as a devel- S_SDSS: Data Services Authorization Object for Functions
opment or production system [page 55]
Establish a connection to the SAP server S_USER_GRP: User Master Maintenance [page 57]
Use the Open Hub interface Open Hub: Administration for RFC Destination [page 45]
Browse metadata in an SAP BW source datastore Browse Metadata for an SAP BW Source Datastore [page
60]
Related Information
To use the Open Hub interface, use the profile S_BI_WHM_RFC profile and the S_RFC_ADM authorization.
The S_BI_WHM_RFC profile contains the necessary authorizations to use the Open Hub interface in SAP Data
Services Agent. Additionally, SAP Data Services Agent needs the S_RFC_ADM authorization to work with the
Open Hub interface.
Purpose: This object includes authorization checks for accessing individual administration functions in
transaction SM59.
Field Values
Activity 03
RFCTYPE T
Related Information
The G_800S_GSE authorization allows SAP Data Services Agent to access ERP hierarchies.
Field Values
Activity 03
Class: Basis
Field Values
Background administrator ID Y
Class: Basis
Field Values
The S_CTS_ADMI authorization allows SAP Data Services Agent to perform CTS operations.
Use: DEV
Field Values
SAP Data Services Agent uses the S_DEVELOP authorization in several ways.
Purpose: This implementation of S_DEVELOP allows SAP Data Services Agent to perform a column search.
Field Values
Activity 03
Purpose: The S_DEVELOP authorization allows SAP Data Services Agent to run generated programs on the
SAP server.
Use: DEV
Field Values
Package $TMP
Activity 01 and 02
Purpose: This implementation allows SAP Data Services Agent to import a table or to search for a table.
Field Values
Object name List of tables and views that a user is allowed to access
Activity 03
The S_DSAUTH authorization gives a user access to SAP Data Services Agent functions.
Field Values
Note
In some SAP NetWeaver versions, this authorization object is named ZDSAUTH or S_SDSAUTH. The
objects are identical except for the name.
Related Information
S_DSDEV is the general authorization object that is the SAP Data Services Agent-specific equivalent of the SAP
S_DEVELOP authorization object.
Field Values
Activity 03
Note
In some SAP NetWeaver versions, this authorization object is named S_SDSDEV or ZDSDEV. The objects
are identical except for the name.
Related Information
S_SDSDEV: SBOP Data Services Authorization Object for Development [page 54]
ZDSDEV: SBOP Data Services Authorization Object for Development [page 58]
The S_DSPGMCHK authorization determines which programs may execute in a production environment.
Use: PROD
Text (Description): SBOP Data Services Authorization Object for program names
Field Values
PROGRAM: ABAP program name Program names that are allowed to be executed in a
production environment
Note
In some SAP NetWeaver versions, this authorization object is named S_SDSPGMCK or ZPGMCHK. The
objects are identical except for the name.
Related Information
S_SDSPGMCK: SBOP Data Services Authorization Object for Program Names [page 55]
ZPGMCHK: SBOP Data Services Authorization Object for Program Names [page 59]
The S_IDOCDEFT authorization allows SAP Data Services Agent to work with IDocs.
Field Values
Activity 03
The S_RFC authorization allows users to execute remote functions on an SAP server.
Field Values
Activity 16
Name of RFC to be protected BAPI, CADR, RFC1, SCAT, SDIF, SLST, SUNI, SUTL,
SDTX, SYST, /SAPDS, RSAB, SDIFRUNTIME, and any other
required function group
Class: Cross-application
Field Values
Activity 03
Field Values
Activity 03
Field Values
Activity 03
The S_RS_ICUBE authorization allows SAP Data Services Agent to access an InfoCube.
Field Values
Activity 03
The S_RS_ODSO authorization allows SAP Data Services Agent to access a DataStore Object.
Field Values
Activity 03
The S_SCRP_TXT authorization allows SAP Data Services Agent to read SAP texts.
Field Values
Activity SHOW
The S_SDS authorization enables you to define whether the SAP system should be treated as a development or
a production system from the perspective of SAP Data Services Agent.
Field Values
Note
In some SAP NetWeaver versions, this authorization object is named ZSDS or S_SDSS. The objects are
identical except for the name.
The S_SDSAUTH authorization gives a user access to the SAP Data Services Agent functions.
Field Values
Note
In some SAP NetWeaver versions, this authorization object is named ZDSAUTH or S_DSAUTH. The objects
are identical except for the name.
S_SDSDEV is the general authorization object that is SAP Data Services Agent-specific equivalent of the SAP
S_DEVELOP authorization object.
Field Values
Activity 03
In some SAP NetWeaver versions, this authorization object is named S_DSDEV or ZDSDEV. The objects are
identical except for the name.
Related Information
S_DSDEV: SBOP Data Services Authorization Object for Development [page 49]
ZDSDEV: SBOP Data Services Authorization Object for Development [page 58]
The S_SDSPGMCK authorization determines which programs may execute in a production environment.
Use: PROD
Text (Description): SBOP Data Services Authorization Object for program names
Field Values
PROGRAM: ABAP program name Program names that are allowed to be executed in a
production environment
Note
In some SAP NetWeaver versions, this authorization object is named S_DSPGMCHK or ZPGMCHK. The
objects are identical except for the name.
The S_SDSS authorization lets you to define whether the SAP system should be treated as a development or a
production system from the perspective of the SAP Data Services Agent.
Note
In some SAP NetWeaver versions, this authorization object is named ZSDS or S_SDS. The objects are
identical except for the name.
Related Information
The S_TABU_DIS authorization allows SAP Data Services Agent to access table data in an SAP system.
Class: Basis
Field Values
Activity 03
SAP Data Services Agent uses the S_TCODE authorization in several ways.
Field Values
Field Values
In addition, you should have access to the contents of the following tables:
• RSDAREA
• RSDAREAT
• RSDCUBE
• RSDCUBET
• RSDODSO
• RSDODSOT
The S_TRANSPRT authorization allows SAP Data Services Agent to access the Transport Organizer.
Use: DEV
Field Values
Activity 01
The S_USER_GRP authorization allows SAP Data Services Agent to establish a connection to the SAP server.
Field Values
User group in user master maintenance User group for the SAP Data Services Agent user
The S_USER_PRO authorization allows SAP Data Services Agent to import an authorization profile.
Use: DEV
Field Values
Activity 03
The ZDSAUTH authorization gives a user access to SAP Data Services Agent functions.
Field Values
Note
In some SAP NetWeaver versions, this authorization object is named S_SDSAUTH or S_DSAUTH. The
objects are identical except for the name.
Related Information
ZDSDEV is the general authorization object that is the SAP Data Services Agent-specific equivalent of the SAP
S_DEVELOP authorization object.
Field Values
Activity 03
Note
In some SAP NetWeaver versions, this authorization object is named S_DSDEV or S_SDSDEV. The objects
are identical except for the name.
Related Information
S_DSDEV: SBOP Data Services Authorization Object for Development [page 49]
S_SDSDEV: SBOP Data Services Authorization Object for Development [page 54]
Use: PROD
Text (Description): SBOP Data Services Authorization Object for program names
Field Values
PROGRAM: ABAP program name Program names that are allowed to be executed in a
production environment
Note
In some SAP NetWeaver versions, this authorization object is named S_DSPGMCHK or S_SDSPGMCK. The
objects are identical except for the name.
S_DSPGMCHK: SBOP Data Services Authorization Object for Program Names [page 49]
S_SDSPGMCK: SBOP Data Services Authorization Object for Program Names [page 55]
The ZSDS authorization lets you to define whether the SAP system should be treated as a development or a
production system from the perspective of SAP Data Services Agent.
Field Values
Note
In some SAP NetWeaver versions, this authorization object is named S_SDSS or S_SDS. The objects are
identical except for the name.
Related Information
To browse metadata for an SAP BW source datastore, access the contents of several tables.
Use the following tables to browse metadata for an SAP BW source datastore:
• RSDAREA
• RSDAREAT
• RSDCUBE
If you do not have access to these tables, request access from your administrator.
Enabling SNC provides a secure connection between SAP systems and the SAP Data Services Agent.
1. Open the Datastores tab and add or select the datastore for which you want to enable SNC.
2. In the Authentication option, select SNC.
Related Information
When you use ABAP transforms in an SAP Cloud Integration for data services data flow, there are additional
configuration options that you need to consider.
In all cases where you use an ABAP transform in SAP Cloud Integration for data services, data is sent via RFC
from the SAP application server to the SAP Data Services Agent. In order to send the data via RFC, you must
first configure the RFC destination in the SAP application server.
When you use an ABAP Query transform in an SAP Cloud Integration for data services data flow, it can be used
in two ways:
This is the recommended execution mode for sandbox and SAP application development
environments.
The ABAP created by the data flow resides on the same host system as the SAP Data Services Agent and
is submitted to the SAP system using the /BODS/RFC_ABAP_INSTALL_AND_RUN function. You should use
this option if the data flow changes each time that it is executed.
• Execute pre-loaded
Tip
The ABAP resides on the SAP application server and is submitted using SAP Data Services RFC function
modules. You should use this option if the data flow does not change each time that it is executed.
In many production environments, the security policy prohibits the execution of auto-generated code.
In this case, the ABAP programs need to be transported to the SAP system manually. The SAP BASIS
administrator can review the ABAP programs prior to uploading, and can add additional security checks.
For more information, see “Uploading ABAP programs to the SAP system”.
When you use a Custom ABAP transform in an SAP Cloud Integration for data services data flow, the generated
ABAP program will contain the custom ABAP FORM. In the datstore, if the ABAP execution mode is set to
Execute pre-loaded, the generated ABAP program needs to be installed on the SAP server.
Related Information
Before you can extract from SAP Business Suite application sources in an SAP Cloud Integration for data
services data flow, you must register the RFC destination in the SAP application server.
In the SAP application server, use transaction SM59 to configure an RFC destination with the following settings:
Note
If you attempt to test the connection with these settings, it is normal for the test to fail due to a connection
timeout. No listener is active unless an SAP Cloud Integration for data services task is currently running.
When you use the Execute pre-loaded datastore option in an ABAP query transform, you must manually upload
the ABAP program to the SAP system.
Before you can run the task in Execute pre-loaded mode, you must first run the task in Generate and Execute
mode on a development system to generate the ABAP program.
Note
The manual process described here can be done automatically using the Generate and view ABAP
report dialog in the data flow editor in the SAP Cloud Integration for data services user interface.
1. Locate the generated ABAP file on the SAP Data Services Agent host system.
2. Copy the contents of the ABAP file.
3. Run transaction SE38 in the SAP system.
4. Create a new program with the name shown as defined in the R3 data flow.
5. Paste the contents of the generated ABAP file into the new program.
Related Information
You must configure the RFC destination including the Program ID to enable loading data from SAP Cloud
Integration for data services to SAP BW.
1. From the SAP Data Warehousing Workbench window, go to Modeling Source Systems External
System .
2. Right-click External System to create a new one.
3. Give the system a name and a description.
4. Click the check mark button, and the RFC Destination window appears.
5. In the Technical Settings tab, select Registered Server Program.
6. Enter the Program ID.
Caution
If you have multiple BW systems, make sure the Program ID values are exactly the same.
Note
The DSOnPremiseAgentXXX.log file contains all agent log files. This section details various parameters and
methods to optimize your log storage to avoid loss of files or memory overload.
Note
The parameters listed below are hidden but configurable within the DSConfig.txt file.
Important: When you modify a parameter's value, you must restart the agent for your change to take effect.
Parameter Description
AgentlogFileCount = 25 By default, there is a limit of 25 files for the log, after which the files wrap
around.
EnableTrace = false By default, this parameter is set to false, which provides minimal infor-
mation. If you want more debugging information, change the value to
True.
Related Information
The agent is scheduled to continuously check and clean out log files that are not needed in order to optimize
space.
The AgentLogRetentionInHours = 720 flag stores all agent log files for a default of 720 hours.
SAP Cloud Integration for data services uses PGP to encrypt or decrypt sensitive data that is stored in files.
PGP provides privacy and security.
By encrypting the files, only the intended receiver will be able to see the actual content. The optional digital
signature verifies the sender's identity. It is recommended that you use PGP to protect all sensitive data.
PGP keys are managed through the Data Services Agent Configuration program. Within an SAP Cloud
Integration for data services organization, a single key pair is shared between all agents. Additionally any
external (third-party) public keys must be imported on all systems hosting an SAP Data Services Agent.
The following keys are used to read files from an external source:
Key Use
Organization private key Used to decrypt the data from the external third-party
External third-party public key Imported and then used to verify the digital signature
Key Use
External third-party public key Used by SAP Cloud Integration for data services to encrypt
data
Organization private key Used when generating the optional digital signature.
Organization public key Exported from SAP Cloud Integration for data services. Sent
to third party to use to verify the digital signature
Related Information
Within an SAP Cloud Integration for data services organization, generate a single PGP key pair.
The key pair contains a public key and a private key. The organization public key can be sent to third-parties
who can use it to encrypt data. SAP Cloud Integration for data services can decrypt the data using the
organization private key.
1. If the SAP Data Services Agent configuration program is not already running. start it.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure PGP.
3. Click Generate a key pair for your organization.
a. Select the key size, hash algorithm, and symmetric algorithm appropriate for your requirements.
b. Enter a user ID.
The user ID is the name bound to the public key. It can be an email address, name, or other identifying
information.
4. Click Apply.
A PGP key pair is generated and saved to the host system where your SAP Data Services Agent is installed.
If your organization has multiple agents, all agents must share the same key pair. The file containing the
organization's PGP key pair must be stored locally on each system that hosts an SAP Data Services Agent.
After the organization's key pair has been generated, it must be exported to a known location and then
imported to each system which hosts an SAP Data Services Agent.
1. If the SAP Data Services Agent configuration program is not already running. start it.
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure PGP.
3. Click Export your organization's key pair.
4. Type or browse to the desired location and type a passphrase.
Take note of this information as it will be required later when you import the key pair.
5. Click Apply.
6. From a system which hosts a different SAP Data Services Agent, start the SAP Data Services Agent
configuration program as described in Step 1.
7. Click Import your organization's key pair.
8. Enter the location and passphrase you created in Step 4 when you exported the key pair from the system
where it was generated.
9. Click Apply.
10. Repeat steps 6 - 9 for each system which hosts an SAP Data Services Agent.
Import an external (third-party) public key to use when encrypting data you are loading to a file.
Note
The external (third-party) public key must be imported to the server hosting the SAP Data Services agent
used in the task.
1. If the SAP Data Services Agent configuration program is not already running. start it.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure PGP.
3. Click Import an external (third-party) public key.
Export your organization's public key so it can be used when encrypting the source data.
1. If the SAP Data Services Agent configuration program is not already running. start it.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
2. Click Configure PGP.
3. Click Export your organization's public key.
4. Type or browse to a location where your public key can be accessed as required.
5. Click Apply.
In order to read and decrypt a PGP-protected source file, your organization's public key must be used to
encrypt the source file.
Additionally, to decrypt a file which contains a digital signature to verify the authenticity of the data's origin and
integrity, you must have the external (third-party) key from the owner of the source file.
As needed for your situation, from the Data Services Agent Configuration program, make sure that the
following prerequisites are met:
❑ A PGP key pair exists for your organization. Generating a PGP Key Pair [page 67]
❑ The organization key pair is imported to the system hosting If the key pair was generated on the
your agent. system hosting your agent, you do not
need to import it.
❑ The owner of the source file has your public key. Export your public key and send it to
the owner of the source file.
❑ The owner of the source file has encrypted the file using your
public key.
Additionally, if the source file contains a digital signature, make sure you have met the following prerequisites:
❑ You have received the external (third-party) public key from the
owner of the source file.
❑ You have imported the external (third-party) public key to the Importing an External Public Key [page
system which hosts your agent. 68]
First use the Data Services Agent Configuration program to meet the prerequisites. Then, use the SAP Cloud
Integration for data services user interface to create and run the task to read and decrypt the source file.
1. In the SAP Cloud Integration for data services user interface, create a task and data flow to read the
encrypted source data.
2. In the data flow, select the transform that reads the source data.
3. In the Transform Details do the following:
a. From the File Options tab, in the Selected input information, in the PGP Protected field, select yes.
b. If the file contains a digital signature, in the PGP Signature field, select yes.
In order to load data to a PGP-protected target file, the public key of the external third-party that will receive the
file must be used to encrypt the source file.
Additionally, to encrypt a file with your digital signature to verify the authenticity of the data's origin and
integrity, you must use your organization's public key.
As needed for your situation, from the Data Services Agent Configuration program, make sure that the
following prerequisites are met:
❑ You have received the public key of the external third-party that Make sure to get the user ID of the key.
The user ID can be an email address,
will receive the target.
name, or other identifying information.
❑ You have imported the external third-party public key. Importing an External Public Key [page
68]
Additionally, to generate your digital signature, make sure you have met the following prerequisites:
❑ A PGP key pair exists for your organization. Generating a PGP Key Pair [page 67]
❑ The organization key pair is imported to the server hosting your If the key pair was not generated on
agent. the server hosting your agent, you must
move it to the server.
❑ You have exported your organization's public key. Exporting your Public Key [page 69]
❑ You have sent your public key to the external third-party that
owns the target.
First use the Data Services Agent Configuration program to meet the prerequisites. Then, use the SAP Cloud
Integration for data services user interface to create and run the task that creates the PGP-encrypted target
file.
1. In the SAP Cloud Integration for data services user interface, create a task to load a target file.
2. Create a data flow. In the Set Up step, in the Encrypt with PGP field, select yes and type the user ID of the
external third-party public key.
3. If you want to include a digital signature, in the Include Digital Signature field, select yes.
Next steps:
Errors may occur during the installation, configuration, or operation of the SAP Data Services Agent. For more
information, see the log files or other available information resources.
If you encounter issues with the SAP Data Services Agent during the installation or configuration processes,
you can check the log files created on the host system for more information.
For more information about troubleshooting common issues regarding SAP Cloud Integration for data services
and the SAP Data Services Agent, see SAP Note 1800845 .
Related Information
The Agent diagnostic tool checks for common issues that cause the Data Services Agent to go offline or that
prevent the agent service from starting. You can attach the information gathered with this tool to SAP Support
tickets for efficient resolution of agent issues. The location of the file to attach is noted at the end of the results.
Before running the agent diagnostics tool, make sure that you have registered the Data Services Agent with
your SAP Cloud Integration for data services server.
You can run the Agent diagnostic tool via a user interface to obtain the following types of information for
analysis:
• System-related information including operating system, IP addresses, processors, JVM memory, and
system space statistics
• Network diagnostics to check communication between the Data Services Agent and SAP Cloud Integration
for data services server
• TCP/IP port information
• Security certificate information
You can also run the Agent diagnostic tool using the command line to obtain an export of an entire repository
as well as a specific task or process. The export can be in ATL format or in XML formatted for Data Services
Designer.
Related Information
You can run the tool via its interface rather than by the command line.
Note
You must run the configuration program from a user account that has administrative privileges. On
Windows platforms that have User Account Control (UAC) enabled, you can also choose the Run as
administrator option.
By default, the configuration program is located in the directory where you installed the SAP Data Services
Agent.
Running the Data Services Agent Configuration Tool via a command line allows you to export a specific task
or process or an entire repository in ATL and XML format for troubleshooting purposes. An ATL file is a
proprietary SAP text file type that contains repository information.
You can add the various generated files from this command line tool to customer support cases, which
provides useful information to SAP.
You can run the Agent Configuration Tool for both sandbox and production (prod) repositories.
For Windows:
For Linux:
Important Usage Notes about the Data Services Agent Configuration Tool
• SAP Cloud Integration for data services does not allow the task or process name to have a space, so the
-name option is always the entire name. There is no need to use quotes.
• In SAP Cloud Integration for data services, a task or process name is case insensitive; therefore, you can
use mixed case in the tool.
• -export or -agentdiagnostic must be the first parameter.
• -export has to be followed by atl or xml.
• -repo has to be followed by sandbox or prod.
• Microsoft Windows is case-insensitive; Linux is case-sensitive.
You can export the details of a task or process in ATL format, which you can attach to an SAP Support ticket for
analysis and troubleshooting.
The Agent Configuration Tool provides timestamp information in the output file name.
Within the SAP Data Services Agent Configuration Tool, enter one of the following:
Option Description
The output path for the exported ATL file is indicated in the tool and will be similar to this example:
C:\ProgramData\SAP\DataServicesAgent\log\RepoOutput\BWInfoPackage_<20210727_2011324
13>_sandbox.atl.
You can export the details of a task or process in Data Services XML format, which you can attach to an SAP
Support ticket for analysis and troubleshooting.
The Agent Configuration Tool provides timestamp information in the output file name.
Within the SAP Data Services Agent Configuration Tool, enter one of the following:
Option Description
The output path for the exported XML file is indicated in the tool and will be similar to this example:
C:\ProgramData\SAP\DataServicesAgent\log\RepoOutput\BWInfopackage_<20210730_452484>
_sandbox.xml.
You can export the details of an agent repository in ATL format, which you can attach to an SAP Support ticket
for analysis and troubleshooting
The Agent Configuration Tool provides timestamp information in the file name.
Within the SAP Data Services Agent Configuration Tool, enter one of the following:
Option Description
The output path for the exported ATL file is indicated in the tool and will be similar to this example:
%DS_COMMON_DIR%\RepoOutput\all_<20210730_452484>_sandbox.atl.
You can export the details of an agent repository in Data Services XML format, which you can attach to an SAP
Support ticket for analysis and troubleshooting.
The Agent Configuration Tool provides timestamp information in the file name.
Within the SAP Data Services Agent Configuration Tool, enter one of the following:
Option Description
%DS_COMMON_DIR%\RepoOutput\all_<20210730_452484>_sandbox.xml.
The Agent Diagnostic Tool supports using the command line to examine the communication between the agent
and the server.
Running the Agent Diagnostic Tool via the command line provides additional functionality compared to the
Run agent diagnostics button that is available within the Configure Agent user interface. Running this by the
command line provides a more granular set of diagnostic information. For example, the command line method
enables you to specifically generate diagnostic logs whereas running the tool from the user interface generates
all diagnostic information, which may take some time depending on the contents of the agent.
The following information is available:
General Information
Privilege Information
System Information
• win
• netstat -av
• tcpip maxuserport/tcptimewaitdely if there is one
• List all the process (tasklist or ps -ef)
• List all the files under %LINK_DIR%\ssl\trusted_certs
The following are only for Linux if available for the user to run:
• lsof
• ulimit -a
Network communication between the agent and server for the C++ part
Simulates Data Services' concatenating the qualified certificates under %LINK_DIR%\ssl\trusted_certs and
allows the curl command to the server.
Agent has C++ and Java code that needs to communicate to the server. C++ is using dsod.pem and Java is
using a Java keystore. This action is to use the Java keystore information to communicate with the server.
Certification Information
If the agent has set up network communication with the server, then the following table applies:
If the agent has not set up network communication with the server, then the following table applies:
Note
There is general information, privilege information, network information, netstat information, Java keystore
information, and all certificates in the Java keystore.
For Windows:
For Linux:
Option Description
If the internal database is still running when you try to uninstall the SAP Data Services Agent, the uninstallation
script may be unable to delete some files.
If the script fails to delete some files, first stop the internal database:
dbstop -y dsod_agent_repo
After stopping the internal database, you can manually delete any remaining files and folders left in the
following locations:
• %LINK_DIR%
• %DS_COMMON_DIR%
• %DS_USER_DIR%
If you encounter errors while uninstalling the SAP Data Services Agent, or have removed the uninstallation
script, you can manually uninstall the software.
1. Close any open files, windows, or command prompts in the %LINK_DIR% or %DS_COMMON_DIR% folders.
If you don't close open files, windows, or command prompts in these locations, you may be unable to
remove all agent files.
By default, the data source is located in ODBC Data Sources System DSN .
5. Uninstall the internal database driver.
regsvr32 /u "%LINK_DIR%"\sqla\dbodbc12DSAgent.dll (prior to SP11 patch 31)
regsvr32 /u "%LINK_DIR%"\sqla\dbodbc17DSAgent.dll (SP11 patch 31 or later)
6. Delete the installed files and folders under %LINK_DIR%, %DS_COMMON_DIR%, %DSOD_APPDATA%
(applicable only to patch 38 or later), and %DS_USER_DIR%.
7. Remove the %LINK_DIR%, %DS_COMMON_DIR%, %DSOD_APPDATA% (applicable only to patch 38 or later),
and %DS_USER_DIR% system environment variables.
To remove the SAP Data Services Agent entry from Programs and
Features in the Windows Control Panel, remove the registry key
HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\SAPDataServicesAgent.
Hyperlinks
Some links are classified by an icon and/or a mouseover text. These links provide additional information.
About the icons:
• Links with the icon : You are entering a Web site that is not hosted by SAP. By using such links, you agree (unless expressly stated otherwise in your
agreements with SAP) to this:
• The content of the linked-to site is not SAP documentation. You may not infer any product claims against SAP based on this information.
• SAP does not agree or disagree with the content on the linked-to site, nor does SAP warrant the availability and correctness. SAP shall not be liable for any
damages caused by the use of such content unless damages have been caused by SAP's gross negligence or willful misconduct.
• Links with the icon : You are leaving the documentation for that particular SAP product or service and are entering an SAP-hosted Web site. By using
such links, you agree that (unless expressly stated otherwise in your agreements with SAP) you may not infer any product claims against SAP based on this
information.
Example Code
Any software coding and/or code snippets are examples. They are not for productive use. The example code is only intended to better explain and visualize the syntax
and phrasing rules. SAP does not warrant the correctness and completeness of the example code. SAP shall not be liable for errors or damages caused by the use of
example code unless damages have been caused by SAP's gross negligence or willful misconduct.
Bias-Free Language
SAP supports a culture of diversity and inclusion. Whenever possible, we use unbiased language in our documentation to refer to people of all cultures, ethnicities,
genders, and abilities.
SAP and other SAP products and services mentioned herein as well as
their respective logos are trademarks or registered trademarks of SAP
SE (or an SAP affiliate company) in Germany and other countries. All
other product and service names mentioned are the trademarks of their
respective companies.