1. Mirror121 Requirements

Mirror121 is a Java based application, designed to mirror data from tables in Salesforce into your local database. This mirroring process is done using Salesforce’s SOAP API and Bulk API.

architecture diagram

What you need:
- Salesforce organization
- Salesforce User with sufficient Rights
- Server with one of supported operating systems - Supported platforms
- Database into which all mirrored data will be stored

1.1. Salesforce

For ideal communication between Salesforce and Mirror121 we recommend you to set up authentication credentials with sufficient rights.

1.1.1. User Access Rights

Mirror121 application communicates with Salesforce via Web Services. We are using API library provided by Salesforce itself. It contains Partner API for synchronous calls and Bulk API for asynchronous calls. This library needs Authentication credentials which can be set in Settings (see Salesforce settings).

1.2. Application Requirements

Mirror121 is a Java web application, meaning it runs centrally on a server, and users interact with it throughout web browser from any computer.

1.2.1. Hardware

Mirror121 runs on any reasonably fast PC. See the minimal recommended hardware configuration.

  • Dual core CPU (Intel or AMD)

  • 2GB RAM

  • 1GB HDD for installation

  • Additional 5GB for log files

These requirements expect that only Mirror121 runs on that machine. You will need to run separate database server for mirror database. Of course you may use your existing infrastructure for that database.

Estimating required storage space

Mirror121 keeps a log of all activities - the estimated storage required for these logs is around 80MB per year.

Mirror121 also creates separate activity log for each synchronization run. The storage required for these logs vary depending on how many synchronizations you have, how often you synchronize them and how many records are being synchronized. Naturally, the longer you use Mirror121 more space for logs is required. See the examples below.

Table 1. Examples of required storage for activity logs
Amount of data Estimated storage required per year

Big (~150,000 updated records) synchronization every day

~ 45 MB per year

Big (~150,000 updated records) synchronization every hour

~ 1 GB per year

Smaller (~10,000 updated records) synchronization every day

~ 3 MB per year

Smaller (~10,000 updated records) synchronization every day

~ 72 MB per year

Then the total storage required per year can be calculated as:

Storage per year = 80MB + (number of big daily synchronizations * 45)MB + (number of small daily synchronizations * 3)MB + (number of big hourly synchronizations * 1000)MB + (number of small hourly synchronizations * 72)MB

1.2.2. Java

Java is a piece of software that has to be installed on your computer. Mirror121 does not work without it. Currently, we support AdoptOpenJDK 11 LTS (HotSpot).

On Windows, an up-to-date version of Java is included in the installation package. On Linux or Mac, you have to install it yourself.

Java Is Provided Free Of Charge

Since Java 11, Oracle (the company that owns and develops Java) has changed licensing terms and support model. This topic is quite complex but the important message is that you don’t have to pay for Java.

There are several distributions of Java available:

  • Please, use Open JDK builds from AdoptOpenJDK. More specifically, use Open JDK 11 LTS - HotSpot.

  • Please, avoid releases of Oracle JDK. These are paid and come with Oracle’s commercial support.

1.2.3. Application server

Mirror121 runs on Apache Tomcat web server which is included in the installation package. We upgrade the Tomcat to the latest version on a regular basis.

1.2.4. Databases

Mirror121 uses two databases.

  • Configuration database - stores metadata, i.e. what to synchronize, when to synchronize, etc.

  • Mirror database - stores data mirrored from Salesforce

Configuration DB and mirror DB may use different database servers. However, the best practice is to have one database server containing two databases. For more information see Database Configuration section.

Table 2. Supported databases
Database Version Comments

Oracle

11g and newer

MySQL

5.5 and newer

Community version MariaDB is also supported

Microsoft SQL Server

2008 and newer

PostgreSQL

9.2 and newer

Snowflake

All versions

Snowflake can be used as a mirror database. It cannot be used as a configuration database.

H2

1.3.172

Embedded file-based database. Suitable only for evaluation.

1.2.5. Browser

Mirror121 runs on all modern web browsers such as Google Chrome, Mozilla Firefox and Internet Explorer 8 or higher.

1.2.6. Supported platforms

Mirror121 is operating system agnostic. It can run on any system which supports Java.

Supported platforms
  • Microsoft Windows XP 32/64 bit

  • Microsoft Windows 7 32/64 bit

  • Microsoft Windows 10 32/64 bit

  • Microsoft Windows Server 2003 32 bit

  • Microsoft Windows Server 2008 R2 64 bit

  • Microsoft Windows Server 2012 R2 64 bit

  • Microsoft Windows Server 2016

  • Linux

2. Installing Mirror121

Mirror121 can be installed either using Windows Installer or manually from a ZIP package. Using the Windows Installer is the most easiest and straightforward way to install and configure Mirror121. On Linux you have to do the installation and configuration manually.

2.1. Prerequisites

You need two databases on your database server - configuration database and mirror database. See Database Configuration chapter.

If you don’t have the databases yet, you can use H2 database which is embedded into Mirror121. This will give you an opportunity to test Mirror121 even before your database administrators create the databases for you. Just note, that H2 is not suitable for production environment.

2.2. Windows Installer

2.2.1. Express installation

  1. Run the Installer - Go to a location to where you downloaded Mirror121 installation file and run it.

  2. First Step - Simply click next.

    welcome
  3. License Agreement - Please go through a license agreement and then confirm by clicking on "I agree".

    license
  4. Selecting the action - Select "Express Install" option and click next.

    type express
  5. Summary - Here you can see summary. It shows installation location, used ports windows service and such.

    express summary
  6. Installing - Window with a progress bar.

    installing
  7. Finish - Final screen. After clicking on "Finish", your default browser will be opened and application itself will load.

    finished

2.2.2. Custom installation

  1. Run the Installer - Go to a location to where you downloaded Mirror121 installation file and run it.

  2. First Step - Simply click next.

    welcome
  3. License Agreement - Please go through a license agreement and then confirm by clicking on "I agree".

    license
  4. Selecting the action - Select "Custom Install" option and click next.

    type custom
  5. Selecting the Location - If you would like to install Mirror121 into a location different to default one, feel free to click "Browse" and select your desired location. Once you are done selecting a location, just click "Next".

    location
  6. Configuring Ports and Host - This screen is here for you to change host name and ports.

    ports host
  7. Selecting the Database - Here you choose can which database you are going to use

    This step is about selecting configuration database. If you wish to read more about mirror db, please see mirror db section

    db choose

    Now you can input information necessary to correctly set mirror database.

    • Database Type: Choose a database you are going to use

    • Server: Address of a server where the database is running

    • Port: A port on which the database is running

    • Database: Database name, SID etc.

    • Username: username of a database user through which the Mirror121 is going to access the database

    • Password: password for that user

      db sql server

      NOTE:
      Postgres, SQL Server - If you wish to use different schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

  8. Setting the Service - If you want to run Mirror121 as a service, leave the checkbox checked. Windows service is good if you want to run Mirror121 automatically after the operating system starts up. You can change the service name. This is useful if you need to install multiple instances of Mirror121.
    You can also change the system account under which the service will be running, e.g. if you create a special account on server just for running Mirror121.

    service
  9. Summary - Here you can see summary. It shows installation location, used ports windows service and such.

    custom summary
  10. Installing - Window with a progress bar.

    installing
  11. Finish - Final screen. After clicking on "Finish", your default browser will be opened and application itself will load.

    finished

2.3. Installation On Linux

  1. Create Mirror121 directory /opt/Mirror121. You are free to choose a different directory.

    mkdir /opt/Mirror121
  2. Download Java and store it into /opt/Mirror121/tmp directory. More specifically, download AdoptOpenJDK 11 LTS - HotSpot.

  3. Download Mirror121 ZIP package and store it into /opt/Mirror121/tmp directory.

  4. Unpack AdoptOpenJDK and Mirror121:

    cd /opt/Mirror121/tmp
    tar zxvf OpenJDK11U-jdk_x64_linux_hotspot_CURRENT-JAVA-VERSION.tar.gz
    unzip mirror121-CURRENT_MIRROR121-VERSION.zip
  5. Copy unpacked directories: copy Java to /opt/Mirror121/jre and Mirror121 to /opt/Mirror121.

    cp -r /opt/Mirror121/tmp/jdk-CURRENT-JAVA-VERSION/. /opt/Mirror121/jre
    cp -r /opt/Mirror121/tmp/mirror121-CURRENT-MIRROR121-VERSION/. /opt/Mirror121
  6. Remove temporary folder

    cd /opt/Mirror121
    rm -rf /opt/Mirror121/tmp
  7. Edit mirror121.properties

    1. mirror121.host property - by default, it is set to "localhost". That means, you can access Mirror121 only from the machine where you installed it. If you want to be able to acces Mirror121 from outside this machine set the property to its network name. See Application URL Configuration for details on how to change the host name.

    2. Provide a connection to your configuration database:

      Database Server config.db.type config.jdbc.url

      SQL Server

      mssql

      jdbc:jtds:sqlserver://<SERVER>:<PORT>/mirror121_config;instance=<INSTANCE-NAME>

      Oracle

      oracle

      jdbc:oracle:thin:@<SERVER>:<PORT>:<SID>
      jdbc:oracle:thin:@<SERVER>:<PORT>/<SERVICE-NAME>

      MySQL

      mysql

      jdbc:mysql://<SERVER>:<PORT>/mirror121_config

      PostgreSQL

      postgres

      jdbc:postgresql://<SERVER>:<PORT>/mirror121_config

      H2 (Embedded)

      h2

      jdbc:h2:~/h2/mirror121_config;MVCC=TRUE

      Also, provide username (config.jdbc.username) and password (config.jdbc.username).

  8. Start Mirror121.

    /opt/Mirror121/run.sh
  9. Open Mirror121 in your favorite web browser on the following URL: \http://<value of mirror121.host property>:<value of mirror121.port>.
     
    The default value is http://localhost:9191.

2.3.1. Service Installation

Optionally, you can install Mirror121 as a service. The service will automatically start when operating system starts.

  1. Open Terminal

  2. Copy a script responsible for starting and stopping Mirror121 init.d directory.

    sudo cp /opt/Mirror121/bin/init.d/mirror121.sh /etc/init.d/mirror121
  3. Make the init script executable:

    sudo chmod a+x /etc/init.d/mirror121
  4. Place symlinks in the run-level directories to start and stop this script automatically.

    1. For Debian based systems:

      update-rc.d mirror121 defaults
    2. For RedHat based systems:

      chkconfig --add mirror121
      The mirror121.sh script contains chkconfig settings

2.4. What Is Installed

We have described the installation process on both Windows and Linux server. Let’s have a look at files we installed.

File & Directory Description

/mirror121.properties

Main configuration file. Contains a connection to the configuration database and Mirror121 base URL. If you want to know more about setting a different base url or changing port, please see chapter Deployment URL

/mirror121-https.jks

Contains a certificate and a private key. Used when you configure Mirror121 to use HTTPS protocol.

/run.bat

Starts Mirror121 on Windows. Used only if a service is not installed.

/run.sh

Starts Mirror121 on Linux.

/shutdown.bat

Stops Mirror121 on Windows. Used only if a service is not installed.

/shutdown.sh

Stops Mirror121 on Linux.

/Uninstall.exe

Uninstalls Mirror121 from Windows.

/Uninstall.info

Stores data needed for uninstallation.

/bin

Contains Apache Tomcat web server.

/conf

Apache Tomcat configuration files.

/jre

Java runtime environment binaries.

/lib

Apache Tomcat classes and libraries.

/license

License files.

/logs

Folder used for both Mirror121 and Tomcat logs.

/logs/activityLogs

Folder used for synchronizations logs. Each synchronization run has its own log file. The files are located by this pattern: <synchronizationId>/<yyyy-MM>/<logId>.log.

/mirror121

Contains all Mirror121 specific application files like configurations, WAR file, documentation and data directory for embedded H2 database and a folder for mirrored attachments.

/mirror121/data

When embedded H2 database is used, this folder stores database files.

/mirror121/docs

Folder contains Mirror121 documentation.

/mirror121/jce

Contains Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files

/mirror121/conf

Mirror121 configuration files. Do not modify. If you need no make a change, modify the main configuration file /mirror121.properties.

/mirror121/webapp

Mirror121 war archive.

/webapps

Used by Apache Tomcat.

/work

Used by Apache Tomcat.

If you want to make sure no directory is missing or you are not sure how Mirror121 installation should look like, please checkout the following directory structure. We omitted directories nested too deep and all files.

.
├── bin
│   ├── experimental
│   └── init.d
├── conf
│   └── Catalina
├── jre
│   ├── bin
│   ├── lib
│   ├── man
│   └── plugin
├── lib
├── logs
│   └── activityLogs
├── mirror121
│   ├── conf
│   ├── data
│   ├── docs
│   ├── license
│   ├── trusted-certificates
│   └── webapp
├── mirror121-themes
├── temp
│   └── attachment
├── webapps
│   └── ROOT
└── work
    └── Catalina

2.5. Uninstall Mirror121

2.5.1. Windows

  1. Go to installation folder.

  2. Open Uninstall.exe and follow the instructions.

2.5.2. Linux

If you have installed Mirror121 manually from an archive file, just follow these steps.

  1. Stop Mirror121

  2. Delete "/opt/Mirror121" directory.

Uninstall Linux Service
  1. Open a Terminal.

  2. Log in as root user.

  3. Remove Service:

    1. For Debian based systems:

      update-rc.d -f mirror121 remove
    2. For RedHat based systems:

      chkconfig --del mirror121

3. Running the Setup Wizard

When you first access Mirror121 you will be redirected to the configuration wizard. It will walk you through basic Mirror121 configuration. Once you finish the Configuration Wizard data will be stored to the database and Mirror121 will be immediately ready to use.

3.1. Walkthrough

3.1.1. License

On this page you can provide a license which you received to your mailbox.

license

3.1.2. Salesforce

Enter information about Salesforce instance. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can authenticate

  • Whether the user is authorized to access all necessary data

If your do not know user’s security token, see Obtaining security token section.
salesforce

3.1.3. Database

Specify a database where all the data mirrored from Salesforce will be stored. Currently, Mirror121 supports Oracle, SQL Server, PostgreSQL and MySQL/MariaDB. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can be authenticated on the given server

  • Whether the user has all necessary privileges

  • Whether the database uses appropriate encoding

database
Postgres, SQL Server - If you wish to use different schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

3.1.4. Snowflake Specific Settings

  • Account Name - Specifies the name of your account. The name is the first part of the URL you use to access Snowflake: https://account-name.snowflakecomputing.com

3.1.5. SMTP Server

Application allows users to subscribe to synchronization notifications. To deliver email to these users the SMTP server must be configured. If you want to enable notifications please specify URL of SMTP server and credentials. You can test the configuration using Test Connection button.

notifications

3.1.6. Security Realm

Mirror121 provides two authentication options.

  1. Internal Database - Users and their credentials are stored in configuration database. You manage the users, their credentials and permissions using Settings page.

  2. LDAP - Users and their credentials are stored outside Mirror121 in your company’s LDAP (most usually Active Directory). You manage the users, their credentials and permission in your LDAP.

Internal Database

If you choose an internal database as a security realm you have to create administrator account. You will use these credentials to login into Mirror121. The account will have all permissions granted.

You can also choose timezone in which all dates will be presented to you. It is a setting of the user interface and does not affect how date are stored in database.

security realm
LDAP

If you choose LDAP as a security realm you have to:

  1. configure a connection to your LDAP server on this page

  2. create Mirror121 roles in your LDAP (see LDAP Groups chapter and LDAP Sample Configuration chapters).

security realm ldap

The meaning of all the field is explained in LDAP chapter.

Always use Test connection button to verify your configuration. If you enter an invalid connection information you lock yourself out and you will not be able to login and change settings. Look at the screenshot above. We clicked the "Test connection" button and we were able to login and the user has a role assigned to him.

3.1.7. Summary

Here you can see a summary of all settings. If you want to change the configuration you can do so by clicking on "Change *" in a particular section.

summary

3.1.8. Finish

After clicking the "Finish" button Mirror121 stores and processes all the settings. The Login page shows up and Mirror121 is ready to use.

login

4. Upgrading Mirror121

4.1. Windows Installer

If you installed Mirror121 using Windows installer you can upgrade to a new version using a new version of Windows installer. Before opening the installer, we recommend you to stop Mirror121. The installer will guide you through the upgrade process, see below.

  1. Download the latest Mirror121 Windows Installer and open it. Then simply click Next.

    welcome
  2. License Agreement - to continue with the installation you have to accept license agreement by clicking on "I agree".

    license
  3. Selecting the action - select "Upgrade an existing Mirror121 installation" option. The installer automatically fills a folder with Mirror121 installation.

    In case you installed Mirror121 to a different location you have to find the folder manually. Click "…​" button and "Browse For Folder" window pops up. Navigate to Mirror121 installation folder and click Ok. Once you are back in the original window, just click Next.

    type upgrade
  4. Summary & Upgrade - review the folder you chose and click Install. When you click Install the upgrade process begins and you can’t revert it back.

  5. Installing - wait for Mirror121 to upgrade. Progress bar shows you how much work has already been done and how much work is remaining.

    installing
  6. Finish - when Final screen shows up it means Mirror121 has been upgraded. After clicking on "Finish", Mirror121 is opened in your default web browser.

    finished

4.2. Manual Upgrade

These upgrade instructions are meant for those who installed Mirror121 manually, i.e. without Windows installer. If you installed Mirror121 using Windows installer you should upgrade it using the Windows installer.

In the following guide, we use "CURRENT-JAVA-VERSION" and "CURRENT_MIRROR121-VERSION" placeholders. You have to replace them with the actual versions of Java and Mirror121.

  1. Stop Mirror121

  2. Download the latest Java and store it into /opt/Mirror121/tmp directory. More specifically, download AdoptOpenJDK 11 LTS - HotSpot.

  3. Unpack AdoptOpenJDK and replace the older version:

    cd /opt/Mirror121/tmp
    tar zxvf OpenJDK11U-jdk_x64_linux_hotspot_CURRENT-JAVA-VERSION.tar.gz
    mv /opt/Mirror121/jre /opt/Mirror121/tmp/jre.bak
    cp -r /opt/Mirror121/tmp/jre<CurrentJreVersion>/. /opt/Mirror121/jre
  4. Download Mirror121 ZIP package and store it into /opt/Mirror121/tmp directory.

  5. Unpack it and run the upgrade:

    cd /opt/Mirror121/tmp
    
    unzip mirror121-CURRENT_MIRROR121-VERSION.zip
    
    /opt/Mirror121/jre/bin/java -cp /opt/Mirror121/tmp/mirror121-CURRENT-MIRROR121-VERSION/mirror121/upgrade/* -Dapplication.logFilePath=/opt/Mirror121/logs/upgrade.log com.mirror121.upgrade.Upgrade /opt/Mirror121 /opt/Mirror121/tmp/mirror121-CURRENT-MIRROR121-VERSION
  6. Remove temporary directory

    rm -rf /opt/Mirror121/tmp
  7. If you installed Mirror121 as a service:

    1. Copy a script responsible for starting and stopping Mirror121 init.d directory.

      sudo cp /opt/Mirror121/bin/init.d/mirror121.sh /etc/init.d/mirror121
    2. Make the init script executable:

      sudo chmod a+x /etc/init.d/mirror121
  8. Start Mirror121

Are you upgrading from Mirror121 older than 3.6.0 and you have changed memory settings in run.sh? Please move your memory settings to mirror121.properties file. By default, Mirror121 is allowed to use up to 2 gigabytes of memory.

5. Settings

5.1. Profile Settings

In Profile Settings page you can see your roles and permissions. You can choose a timezone in which Mirror121 will present dates and times. Note that this settings will affect only your user account.

You can also choose a new password by clicking Change Password button and following a subsequent form.

5.2. Notification Management

To track synchronizations, you can add notifications. Mirror121 provides notifications for user-specified synchronizations when their run fails or the resulting state changes. Each notification can have its own set of tracked synchronizations and recipient email address.

5.2.1. Configure SMTP Server

For Mirror121 to be able to send notifications SMTP server has to be configured. On the page Settings / Notifications Management / SMTP Server you can specify its URL, port and credentials and security option.

Security option defines if and how the communication with SMTP server is encrypted:

  • None - No encryption on the outgoing emails. They will be sent as plain as day between Mirror121 and the server.

  • STARTTLS - Mirror121 will ask the SMTP server if it’s it’s possible to send data using TLS. If so, it will use the secure method. If not, data will be sent unencrypted.

  • SSL - Mirror121 will send data to SMTP server using encrypted connection. Data is never send unencrypted. This is the most secure method.

notifications smtp

At the bottom of the page there is a button called Test Connection. It validates the configuration and send testing email to the email address you provide it the dialog window.

notifications smtp tester

5.2.2. Create a New Notification

To create a new notification you need to input an e-mail address. You can also choose specific synchronization you want to be informed about. Then you will choose what events you want to be notified about.

  • Every state change - you’ll be notified only when the state of the synchronization changes

  • All failed, all warnings, first successful - you will be notified after every failed or warning run. When you fix the synchronization, you’ll be notified too.

  • All failed, first warning, first successful - you will be notified after every failed run. When the synchronization ends with warning or you fix it, you’ll be notified too.

  • Every synchronization run - notification will be sent after every synchronization run regardless its state

notifications add

5.2.3. Delete Notification

All added notifications are in a notification table. To cancel sending of notifications you can delete them by selecting a checkbox next to desired email address and clicking on Delete button.

notifications delete

5.3. User Management

User Management page allows you to configure which users will be allowed to login and which permissions will be assigned to them.

There are two authentication options: internal database or LDAP.

This setting option is available only for users with 'Synchronization administrator' role. See User Roles.

5.3.1. Security Realms

  • Internal Database - Users and their credentials are stored in configuration database. You manage the users, their credentials and permissions using Settings page.

  • LDAP - Users and their credentials are stored outside Mirror121 in your company’s LDAP (most usually Active Directory). You manage the users, their credentials and permission in your LDAP.

  • SSO - Single Sing-On using SAML 2.0

5.3.2. Internal Database

Choose "Internal Database" in the Security Realm field. Then, don’t forget to create a user Create New User with administrator permissions before you logout. Otherwise, you’d lock yourself out and would not be able to login and change settings.

security realm internal

5.3.3. LDAP

LDAP configuration is a bit more complicated than Internal database so let’s look at all the attributes in detail.

security realm ldap
Field Description

Server

URL of your LDAP server. For example: myLdapServer.mirror121.com

If your LDAP server uses a port other than default 389 (i.e. standard LDAP port) you can also append a port number. For example: myLdapServer.mirror121.com:1234

To connect to LDAP over SSL, specify it with the ldaps:// protocol. For example: ldaps://myLdapServer.mirror121.com

If your LDAP over SSL uses a port other than default 636 (i.e. standard LDAPS port) you can also append a port number. For example: ldaps://myLdapServer.mirror121.com:1234

Manager DN

If your LDAP server does not support anonymous binding (i.e. it does not allow you to send a query without authentication), then you have to provide Manager credentials. Mirror121 will authenticate using these credentials and will query for user and group data using this account.

DN identifies the Manager user. For example: CN=MyUser,CN=Users,DC=myDomain,DC=com

Manager Password

Password of the user from Manager DN field.

Root DN

LDAP is basically a tree. If empty we will search the whole three. To improve performance you can specify the base DN - i.e. subtree. Then we will search only in this subtree. For example: DN=myServer,DN=com

User Search Base

Where in LDAP tree to search for users.

If empty, Mirror121 will search the whole tree.

If not empty, Mirror121 will search only a subtree. It is relative to Root DN. For example: CN=Users,CN=Employees

User Search Filter

The query used for finding users by their username.

Defaults to "(sAMAccountName={0})" - i.e. user name is stored in "sAMAccountName" attribute.

If you want to allow access only to user in a group then type: "(&(sAMAccountName={0})(memberOf=CN=Your CN,OU=Your OU,DN=YourDomain,DN=com))" and change "memberOf" to match the full name of your desired group.

Group Search Base

Where in LDAP tree to search for group your users belong to.

If empty, Mirror121 will search the whole tree.

If not empty, Mirror121 will search only a subtree. It is relative to Root DN. For example: CN=Mirror121,CN=Applications

Group Search Filter

Looks for LDAP groups to which a user belongs to.

Defaults to "(member={0})" - i.e. it looks for all groups which have a "member" attribute set to DN of the user.

Some LDAP implementations use "uniqueMember" attribute instead of "member" attribute.

Allowed placeholders:
{0} - gets replaced by DN of the user
{1} - gets replaced by the username of the user

Group Role Attribute

The ID of the attribute which contains the role name for a group. Default value: cn

After you fill in all the fields you should test your configuration using "Test Connection" button. This is really important because if the configuration is invalid you would lock yourself out and won’t be able to login to Mirror121 at all.

After Test connection popup window opens, provide credentials of the user you will use to login to Mirror121. We will test if the user can login and what permissions are assigned to him.

security realm ldap test popup

If everything went fine you will see a green message. If something goes wrong we print an error message.

security realm ldap test message
LDAP Groups

To be able to work with Mirror121 you have to create groups in your LDAP and assign user to them.

You have to create following groups. Names of the groups match the names of Mirror121 roles. Their meaning is explained in User Manual.

  1. SUPER_ADMINISTRATOR

  2. SYNCHRONIZATION_ADMINISTRATOR

  3. SYNCHRONIZATION_SUPERVISOR

  4. API_CALLER

  5. SYNCHRONIZATION_READER

Choosing Your Own Group Names

If you don’t want to create groups with these exact names, you can choose your own names.

Open "Settings → Advanced Settings" page and modify following properties so that their values match names of your LDAP groups:

  1. mirror121.auth.ldap.role.mapping.SUPER_ADMINISTRATOR

  2. mirror121.auth.ldap.role.mapping.SYNCHRONIZATION_ADMINISTRATOR

  3. mirror121.auth.ldap.role.mapping.SYNCHRONIZATION_SUPERVISOR

  4. mirror121.auth.ldap.role.mapping.API_CALLER

  5. mirror121.auth.ldap.role.mapping.SYNCHRONIZATION_READER

Fallback Role

Mirror121 does not allow users to login if they have no role assigned. This behavior is configurable - if a user does not have any role, you can assign him a default role.

You can do that by putting Mirror121 role name into "mirror121.auth.ldap.role.fallbackRoles" advanced property.

Sample LDAP Configuration

It does not matter where in LDAP tree you create the groups. See an example from our testing LDAP server:

security realm ldap sample users and roles

We have two users in our company. We have created the groups inside "ou=Mirror121,ou=Employees,ou=Company" subtree.

Let’s look at the detail of SYNCHRONIZATION_ADMINISTRATOR group. See the member attribute? In this example, John Smith was assigned to this group. When he logs is he is able to do all the action super administrators can do.

security realm ldap sample role detail

5.3.4. Single Sing-On (SSO)

Mirror121 supports Single Sing-On over SAML 2.0 protocol and Web SSO profile.

Prerequisites
  1. You have your own Identity Provider (IdP) up and running.

  2. In case your IdP runs on HTTPS Mirror121 has to run on HTTPS too.

Dictionary
  • IdP: Identity Provider is an application which authenticates users

  • SP: Service Provider is an application which uses IdP to authenticate user. Mirror121 is a service provider.

  • Entity Id: Unique identification of an application in IdP and SP.

Setup
  1. IdP: Register Mirror121 as a new service provider. The service provider is sometimes referred to as a client. By default, Mirror121’s Entity Id (sometimes referred to as "Client Id" or "Service Provider Name") is Mirror121 but you can choose whatever name you want.

  2. IdP: locate IDPSSODescriptor. The location is IdP-specific. See documentation of your IdP to learn where to find it.

    IDPSSODescriptor describes the IdP server:

    • URLs where to reach the IdP

    • Information wherether Mirror121 should sign requests

    • Public key of the IdP (which allows Mirror121 to verify IdP’s signatures)

      See sample IDPSSODescriptor. You cannot use this samle in your Mirror121 installation. Instead, you have to get your own from your IdP. The sample should give you an idea what kind of file Mirror121 needs.

  3. Mirror121: Configure Mirror121 to use SSO. Go to Settings → User Management → Security Realm page and choose "SSO" as a "Security Realm"

    • Entity Id: You have registered Mirror121 in your IdP. Please use the same name here. Default value is Mirror121.

    • IdP Metadata: Copy and paste the IDPSSODescriptor from step 2. Note that if you change a configuration in IdP the "IDPSSODescriptor" may change too. Always make sure Mirror121 uses up-to-date descriptor.

    • Signing key: A pair of public and private key.

      Mirror121 uses the private key to sign data. IdP uses the public key to verify the signature. IdP knows the public key because it is available on "/saml/metadata" page.

      By default, Mirror121 uses self-signed certificate but you can import and use your own certificate.

    • Encryption key: A pair of public and private key.

      IdP uses the public key to encrypt assertions (it knows the public key because it is available on "/saml/metadata" page). Mirror121 decrypts the assertions with the private key.

      By default, Mirror121 uses self-signed certificate but you can import and use your own certificate.

    • Roles Source: Mirror121 loads the roles from its own configuration database and ignores (if provided) the roles in IdP.

    • Require Local Account: If enabled, a user can login only if a matching account exists in Mirror121’s configuration database. When installing Mirror121, you setup an admin account stored in configuration database. Using that account you setup user accounts for SSO users and assign them roles before they login. When you properly configure SSO and setup admin account you can disable login using the initial admin account.

    • Internal Realm: When enabled you can login using credentials from internal database. We recommend you to have Internal Realm enabled until you successfully finished SSO configuration.

      • Enabled - besides SSO, you can login on /sideDoor page using credentials from the configuration database. This is especially useful when you’re testing SSO configuration. If the SSO configuration is invalid you can still login using the config. database credentials and you won’t get locked out.

      • Disabled - you can login only using SSO. If the SSO configuration is invalid you’re locked out.

    • Once the configuration is finished and saved, SSO endpoints are made public and metadata file is generated.

      security realm sso

      The configuration page in Mirror121 does not contain any "Test Connection" button. It may be convenient to be logged in in one browser using credentials from configuration database and test the configuration in another browser (not just a tab). This way you’re always logged in in one browser and you use the second one for testing.

  4. IdP: Finish the configuration you started in the first step. Use the information below.

Using Your Own Signing And Encryption Keys

To setup signing and encryption Mirror121 need a private key and public certificate. By default, Mirror121 contains self signed certificate. To use your own you have to import it into Mirror121.

  1. Download and install Keystore Explorer {link-keystore-explorer}. Keystore Explorer is a 3rd party tool which allows you to import keys and certificates. You could use a command-line "keytool" which is bundled with Mirror121 but Keystore Explorer is simpler to use.

  2. Open mirror121/data/sso/samlKeystore.jks in KeyStore Explorer and import signing key/certificate pair (they’re bundled in one file). Then import encryption key/certificate pair. Use default password "changeit".

  3. Go to Settings → User Management → Security Realm and select the keys in "Signing Key" and "Encryption Key" fields

ADFS

If you use ADFS (Active Directory Federation Services) you have to configure Mirror121 and ADFS as follows:

  • Mirror121: Go to Settings → Advanced Settings page. Find "mirror121.auth.sso.webSSOProfileOptions.includeScoping" property and set it to false.

  • ADFS: Add NameID as "Claim rule name", choose "Active Directory" as Attribute store, choose "SAM-Account-Name" as LDAP Attribute and "Name ID" as "Outgoing claim type", finish the wizard and confirm the claim rules window, in ADFS 3.0 you might need to configure the Name ID as a Pass Through claim.

  • ADFS: Open service provider by double-clicking it, select tab Advanced and change "Secure hash algorithm" to SHA-1

How To Configure Cluster

This chapter is intended for owners of Cluster license. If you have a Cluster license you have multiple Mirror121 installations (nodes) using the same configuration database.

Following properties must be configured separately for each node: Entity Id, Encryption Key, Signing Key.

If you log into the first node and open Settings → User Management → Security Realm page you can configure Entity Id, Encryption Key, Signing Key of that node. If you log in to the second node you configure the second node.

Properties other than Entity Id, Encryption Key, Signing Key are shared between the nodes and it does not matter from which node you configure them.

You should restart Mirror121 after you make a change. That is necessary so that both nodes are properly initialized and aware of that change.

5.3.5. Create New User

If you use Internal database security realm you can create a new user. You have to provide its username, password, timezone and roles. See User Roles chapter so that you can decide which role you want to assign to the user. There is usually no need to add more than one role (except for the API Caller role).

user add

5.3.6. User Detail

On user detail page you can see user’s information summary - his or hers username, selected timezone, roles and permissions.

Permissions are here just for your information and can’t be altered. They are connected to user’s role not the user itself.
Edit User

On edit page you can edit username, selected timezone and user’s role (See User Roles). If you need to change user’s password see Change Password Section.

Activate/Deactivate

The user’s account can be deactivated by clicking on Deactivate button. After that the user is no longer able to log in. The account can be reactivated by clicking on Activate button.

Change Password

On this page you can set the password to any user. You can find it helpful when someone forgets the password and needs a new one.

5.4. General Settings

5.4.1. Cache Management

The application uses cache for retrieving Salesforce meta-data (what tables Salesforce works with, which columns they have etc.). Because of caching some data could be out of date. Typical example is list of Salesforce tables available to synchronize. Default reload interval of cache is 24 hour.

If you want to clear the cache manually go to Settings menu, choose General Settings and click on Clear All Caches button.

5.4.2. General

  • Use Temporary Tables - This settings applies only to Clean And Synchronize. If it is disabled Mirror121 cleans all data and then downloads them. Because of that you have to wait for you data to be downloaded to mirror database. If it is enabled Mirror121 downloads all data to temporary table. After synchronization finishes it removes old data and moves temporary table to your mirrored table. During Clean And Synchronize you have all your old data available in mirror database.

  • Column Name Suffix - Column names in Salesforce may end with a suffix. For example, custom columns end with "__c". There are two ways how to treat the suffixes.

    • Preserve - column names in your mirror database contain the suffix

    • Remove - column names in your mirror database do not contain the suffix. All such columns can be found on Edit Mapping page.

  • Records Delete - Mirror121 is able to find out when a record is deleted in Salesforce and then delete it in your mirror database. Records Delete allows you to choose how to delete the record.

    • Hard - record is irreversibly deleted from mirror database.

    • Soft - record is not deleted and "mirror_deleted_on" column is filled in. The column contains date and time when Mirror121 found out it was deleted.

      Mirror121 deletes all records (including soft-deleted records) in case:

      • Clean and synchronize is called

      • Truncate delete strategy is used

      • A new column is added using Auto Schema Update

      If you switch from Soft to Hard, "mirror_delete_on" column will be dropped during the next synchronization run.

  • Synchronize Archived - Salesforce archives records in some tables. Those records are hidden but are still available in Salesforce.

    • Yes - Mirror121 synchronizes archived records to the mirror database.

    • No - Mirror121 does not synchronize archived records. When Salesforce archives a record, Mirror121 deletes it from the mirror database.

      Warning: changing this option affects all synchronizations that may contain archived records (Announcement, ContentDocument, Event, CollaborationGroup, Pricebook2, Task). When you switch to Yes, you should Clean & Synchronize affected synchronizations. When you switch to No, you should run Differential synchronization or Clean & Synchronize on affected synchronizations.

  • Synchronize Deleted - Salesforce has a recycle bin which contains deleted records.

    • Yes - Clean synchronization synchronizes even records from recycle bin. This option is useful only if you use Soft deletes.

    • No - Clean synchronization does synchronize records from recycle bin.

      changing this option affects all synchronizations that may contain archived record. When you switch to Yes, you should Clean & Synchronize affected synchronizations. When you switch to No, you should run Differential synchronization or Clean & Synchronize on affected synchronizations.
general

5.4.3. Activity Log Retention

Activity Log Retention removes old activity logs after the specified number of days (30 days by default). It removes activity logs from both configuration database and filesystem. Removing logs from filesystem saves you disc storage whereas removing logs from configuration database improves its performance. This is particularly true if you have a lot of synchronizations running very often.

If you are upgrading from Mirror121 < 3.2.0 this feature is disabled. On new installations it is enabled. You can change the settings on General Settings page.

  • Enabled - enable or disable activity log retention

  • Cleanup database - you may choose to remove the logs from configuration database or leave them there intact

  • Retention period - how many days you want to keep the activity logs intact

activity log retention

5.4.4. Consistency Check

Allows you to configure consistency check. On fresh installations the consistency check is enabled by default. On existing installations it is disabled.

  • Consistency Check - enable or disable consistency check

  • Issue Warnings - If enabled Mirror121 will set status of a synchronization to Warning if it contains suspicious number of records in mirror database. If disabled a result of the consistency will be only written to the activity log and the synchronization will be marked as successful.

consistency check

5.4.5. Skip Failing Records

If enabled Mirror121 will skip the records which it failed to insert or update. If that happens the synchronization will finish with a warning (you can configure notifications not to miss any warning).

If Mirror121 skipped a record you have to analyze the root cause of the issue and fix it. Then you download the missing records using Differential synchronization, Synchronize from or Clean and synchronize.

There is a limit of how many records will Mirror121 skip at maximum. Let’s say you have a table with hundreds of thousands records and Mirror121 fails to insert all of them. Then you want the synchronization to fail also because there must be a global issue (like failing database connection, permissions in Salesforce, etc.). On the other hand if just one record fails then you want to end up with a warning and fix the one records separately.

skip failing records

5.4.6. Timezone Settings

By default Mirror121 stores data to mirror database in UTC. UTC is the most convenient and recommended way to store dates and times especially if your organization spans across several timezones. If you need to access and see date and time fields formatted in specific time zone, Mirror121 provides two ways to achieve that:

  • Mirror Data Time Zone - select a timezone of your choice. Mirror121 still receives dates and times from Salesforce in UTC but when storing them to database it converts them to the timezone of your choice.

  • Timezone Data Types - If you enable this option you will force Mirror121 to use following data types for storage:

    • Oracle - timestamp with local time zone

    • PostgreSQL - timestamp with time zone

    • MySQL - timestamp

    • Embedded H2 - datetime

    • SQL Server - datetimeoffset

      Internally dates and times are stored in UTC (MySQL, SQL Server, PostreSQL) or in database timezone (Oracle). Every time you request the data using SQL they are converted to user local session time zone you chose when creating a connection to mirror database. This does not work in SQL Server where you have to convert the dates from UTC to other time zones manually.

      Also note that you have to drop existing table for this option to work properly - Mirror121 needs to recreate the table with the proper data type.

timezone

5.5. License

On this page you can see basic information about current license.

license

If you click to the Enter License button you can enter a new valid license key. This action overrides the old license key if any is present.

5.5.1. License key

License key is generated string (about 600 characters long) which includes information about Mirror121 application. If you don’t have any key yet please visit our website.

Salesforce

Enter information about Salesforce instance. You can test the configuration using Test Connection button.

Test Connection button checks:

  • Whether the user can authenticate

  • Whether the user is authorized to access all necessary data

If your do not know user’s security token, see Obtaining security token section.
salesforce

5.6. Mirror Database

Allows you to set information about the mirror database (the database where all data from Salesforce is stored). The common scenario is to use your own database server. Built in database is recommended for testing purposes only.

Test Connection button checks:

  • Whether the user can be authenticated on the given server

  • Whether the user has all necessary privileges

  • Whether the database uses appropriate encoding

database mysql
If you want your Oracle or PostgreSQL db to treat identifiers as case sensitive (using of double-quotes in sql statements), please make sure to have Quote Identifiers checkbox checked as seen on picture below.
database oracle
Postgres, SQL Server - If you wish to use other schema than public, it is possible too. Just fill the field Schema with the name of your desired schema.

5.7. Import / Export

In this section you can export your synchronization settings or import your previously exported setting. Note that only settings is exported - the actual data in mirror database stay intact.

Example: You have testing and production environment. Now you want to move all synchronization settings from your testing environment to production environment. You export the settings on testing environment and then import on production.

In case imported file contains synchronizations which already exist Mirror121 will let you choose how to resolve the conflict:

  • Do not import duplicate synchronizations - this option won’t import synchronizations which already exist

  • Automatically rename duplicate synchronizations - this option will import all synchronizations but those with conflicted names will be renamed. For example: you try to import synchronization for table "task" but it already exists - Mirror121 will import the synchronization but its name and name of table in mirror database will be "task_1"

  • Update existing synchronizations - this option will overwrite (update) existing synchronizations.

5.8. Visual Theme

Mirror121 can be configured to use custom visual style such as colors and text next to the logo.

  • Theme color: You can change color of main navigation bar (right under the logo) and all table headers

  • Text Beside Logo: This option will help you to distinguish your Mirror121 instances by putting a text next to a logo (E.g.: development, test etc.). Are you can put here a short message to all Mirror121 users (Max 100 characters allowed)

visual theme

5.9. Advanced Settings

5.10. Time Zones

Every Mirror121 user can set his or hers own time zone. This does not affect communication with Salesforce, or any of Mirror121 databases. The time zone setting is just for presenting data via Mirror121 gui. List of available time zones is configurable.

All data in mirror and configuration database are stored in UTC.

timezone current

Communication with Salesforce happens in two time zones. While requesting data via web service is used UTC (both request and response). However while sending data via web service is used the time zone of Salesforce user. This time zone can be configured in Advanced Settings.

5.10.1. Changing Timezone

For changing timezone click on the planet icon next to timezone in the left bottom.

timezone change

6. Salesforce Information

6.1. User rights

Here you can find steps to create user with sufficient rights for synchronizations. Login to your Salesforce instance using administrator account and follow these steps:

  1. Create Profile (Classic: Setup → Manage Users → Profiles, Lightning: Setup → Users → Profiles)

    1. New Profile

      1. Existing profile = Read Only

      2. Profile Name = <profile name>

      3. Create

    2. Edit <profile name>

      1. Check API Enabled (should be)

      2. In Standard Object Permissions check Read and View All by all tables that you want to synchronize by this user

      3. Save

  2. Create User (Classic: Setup → Manage Users → Users, Lightning: Setup → Users → Users)

    1. New User

      1. User Licence = Salesforce

      2. Profile = <profile name>

      3. Check Active (should be)

      4. Save

6.2. Obtaining security token

Login to your Salesforce instance and follow next steps:

6.2.1. Classic

  1. In the upper menu click on your name and My Settings.

  2. In the left menu click on Personal and Reset my security token.

  3. Click on button Reset my security token and you will have it in your mailbox.

6.2.2. Lightening

  1. In the the upper menu click on icon on the right and then choose "Settings".

  2. In the left menu navigate to "My Personal Information" → "Reset my security token".

  3. Click on Reset Security Token button and you will have it delivered in your mailbox.

6.3. Getting your Salesforce organization ID

Login to your Salesforce instance and follow next steps:

6.3.1. Classic

  1. In the upper menu click Setup.

  2. In the left menu click on Company Profile and Company Information.

  3. On displayed page you have your Salesforce.com Organization ID on the right side of the page

6.3.2. Lightening

  1. In the upper menu click the "Gear" icon and then choose "Setup"

  2. In the left menu navigate to "Company Settings" → "Company Information".

  3. You can find the "Salesforce.com Organization ID" on the right side of the page.

7. Advanced configuration

Most of the configuration settings can be done in Mirror121 on Settings page when the configuration choices are stored in a database. Yet some configuration have to be done using mirror121.properties file.

mirror121.properties

The file is located in <Mirror121InstallationFolder> folder and contains deployment URL settings and configuration database settings.

Settings page

These properties are stored in database in a table called config. You will set the most important settings through Configuration Wizard which pops up when Mirror121 starts for the first time. Later on if you need to change the properties you will do that on Settings page in Mirror121.
Contains all other settings including mirror database, SMTP server, notifications, Salesforce, users, etc.

The mirror121.properties file contains just a small subset of all configuration options. Yet the contents of the file is very important for Mirror121 to run properly. Following subsections explain the mirror121.properties settings in detail.

7.1. Database Configuration

Mirror121 application uses two distinct databases.

  • mirror121_config database - used for storing application synchronization settings, user settings, logs…​

  • mirror database - contains the actual tables synchronized from Salesforce.

This chapter deals with configuration database (mirror121_config) only.
Both configuration and mirror databases have to use UTF-8 encoding. If they don’t Mirror121 may fail to run some synchronizations because it won’t be able to store some characters into the database.

Application supports variety of database systems which can be used for both mirror121_config or mirror database. The actual databases are discussed in following subchapters.

  • mirror121_config database configuration is located in mirror121.properties file. If you want to use a database system other than H2, you can change it during the installation or later in the file manually. Example configuration can be found in the following sections.

  • mirror database is configured via Configuration Wizard. Later on you can change the settings directly in Mirror121 on the page Settings / Database

Mirror121 needs following database permissions:

  1. be able to connect and login to the database

  2. be able to create, alter and delete tables and views

  3. be able to create, alter and delete indexes on those tables

  4. be able to insert, update and delete data in those tables and views

7.1.1. H2

{{link-h2-web}[H2 Database Engine] is a lightweight open source Java SQL database. We have embedded H2 into Mirror121.
The database is created automatically by Mirror121 based on connection url information from mirror121.properties file.

Sample settings of configuration database in mirror121.properties
# Mirror121 configuration database (H2)
config.db.type = h2
config.jdbc.url = jdbc:h2:~/h2/mirror121_config;MVCC=TRUE
config.jdbc.username = sa
config.jdbc.password = sa

7.1.2. MySQL

When creating a new database please use utf8 encoding and utf8_unicode_ci collation.

Sample Create Database SQL
CREATE DATABASE mirror121_config CHARACTER SET = utf8 COLLATE utf8_unicode_ci;
CREATE DATABASE mirror CHARACTER SET = utf8 COLLATE utf8_unicode_ci;
Sample settings of configuration database in mirror121.properties
# Mirror121 configuration database (MySQL)
config.db.type = mysql
config.jdbc.url = jdbc:mysql://127.0.0.1:3306/mirror121_config
config.jdbc.username = root
config.jdbc.password = password

7.1.3. SQL Server

When creating a new database please choose a collation that suits your needs the best.

Sample settings of configuration database in mirror121.properties
# Mirror121 configuration database (SQL Server)
config.db.type = mssql
config.jdbc.url = jdbc:jtds:sqlserver://127.0.0.1/mirror121_config;instance=SQLEXPRESS
config.jdbc.username = root
config.jdbc.password = password
config.jdbc.schema = dbo

7.1.4. Oracle

When creating a database please make sure to use AL32UTF8 character set.

Sample settings of configuration database in mirror121.properties
# Mirror121 configuration database (Oracle)
config.db.type = oracle
config.jdbc.url = jdbc:oracle:thin:@127.0.0.1:1521:mirror121_config
config.jdbc.username = root
config.jdbc.password = password

7.1.5. PostgreSQL

When creating a new database please use UTF8 encoding. Values for lc_collate and lc_ctype properties are provided automatically. If you need to change them please refer to the official PosgreSQL documentation.

Sample Create Database SQL
CREATE DATABASE mirror121_config ENCODING 'UTF8';
CREATE DATABASE mirror ENCODING 'UTF8';
Sample settings of configuration database in mirror121.properties
# Mirror121 configuration database (PostgreSQL)
config.db.type = postgres
config.jdbc.url = jdbc:postgresql://127.0.0.1:5432/mirror121_config
config.jdbc.username = root
config.jdbc.password = password
config.jdbc.schema = public
If you are having performance issues please check out "mirror.springframework.ignoreParameterType" advanced property (Settings → Advanced Settings). If enabled we won’t try to retrieve columns meta-data for every column hence improving the speed of inserts/updates. We recommend having this property enabled for PostgreSQL.

7.2. Application Deployment URL

Deployment address depends is the address of the server where the application is deployed. The application needs to know this address to correctly serve its pages and content.

Provided configuration is used to construct absolute URLs for resources files like JavaScript and CSS. See how Mirror121 constructs URLs using these properties:

Constructed absolute URL
{mirror121.scheme}://{mirror121.host}:{mirror121.port}{mirror121.context}
Example of Application URL Configuration - excerpt from mirror121.properties
### Mirror121 base URL ###
mirror121.scheme = http
mirror121.host = localhost
mirror121.port = 9191
mirror121.controlport = 8006
mirror121.context =
Specifying a context

If you want Mirror121 to run not on http://localhost:9191 but on http://localhost:9191/mirror121 you have to change the mirror121.context property. Note that when you specify a context it has to start with a backslash.

mirror121.context = /mirror121
Control Port

Mirror121 runs on Apache Tomcat server which uses a notion of a control port. The port is used to receive shutdown request. The default value is provided but if you change the port in Apache Tomcat configuration files you also have to change mirror121.controlport property.

mirror121.controlport = 8006

7.3. Security

7.4. Mirror121 Over HTTPS

By default, Mirror121 is accessible through HTTP protocol. That means the communication between the user and Mirror121 is not encrypted. This chapter explains how to enable HTTPS protocol.

  1. Stop Mirror121

  2. Replace HTTP configuration files with HTTPS configuration files. Go to <Mirror121InstallationDirectory>/conf and rename:

    1. server.xml → server-http.xml

    2. server-https.xml → server.xml

    3. web.xml → web-http.xml

    4. web-https.xml → web.xml

  3. Open mirror121.properties file and change "mirror121.scheme" and "mirror121.port" properties.

    mirror121.properties excerpt
       mirror121.scheme = https
       mirror121.port=9443
  4. Now, Mirror121 is configured to use HTTPS. But you need to do one more step - you have to import your certificate and its private key to Mirror121.

    The certificate and its private key are bundled in a single PFX file. You need to import it to mirror121-https.jks file.
    If you are asked for a source keystore password, type your password. If you are asked for a destination keystore password, type "password". When asked if you trust the certificate, type "yes".

    Windows command line
    cd "C:\Program Files\Mirror121"
    del mirror121-https.jks
    jre\bin\keytool -importkeystore -srckeystore <path-to-your-pfx-file> -destkeystore mirror121-https.jks -destkeypass password
    Linux command line
    cd /opt/Mirror121
    rm mirror121-https.jks
    jre/bin/keytool -importkeystore -srckeystore <path-to-your-pfx-file> -destkeystore mirror121-https.jks -destkeypass password
    As you can see, we use a default password. You can choose whatever password you want. If you do so you also have to change it in "server.xml" file. Look for a "keystore" attribute and change its value to your password.
  5. Start Mirror121

7.5. Security Properties

On Settings → Advanced Properties page we offer several security related properties. Usually, there is no need to change them. This chapter explains these properties.

Property Default Description

mirror121.security.httpHeaders.useXFrameOptions

true

If enabled, pages contain "X-Frame-Options: SAMEORIGIN" HTTP header.

mirror121.security.httpHeaders.useXXSSProtection

true

If enabled, pages contain "X-XSS-Protection: 1; mode=block" HTTP header.

mirror121.security.httpHeaders.useXContentTypeOptions

true

If enabled, pages contain "X-Content-Type-Options: nosniff" HTTP header.

mirror121.security.httpHeaders.strictTransportSecurity

false

If enabled, pages contain "Strict-Transport-Security: max-age=31536000 ; includeSubDomains" HTTP header.

mirror121.security.showExceptionOnErrorPages

true

If enabled, the error page will display stack trace of an exception. Disable this property if your security policies do not allow to reveal such information.

mirror121.security.showSystemInfoOnErrorPages

true

If enabled, the error page will display system information like Mirror121 version, database type and version, etc. Disable this property if your security policies do not allow to reveal such information.

mirror121.security.showVersion

true

If enabled, pages contain Mirror121 version in a footer.

mirror121.security.useChartsExternalLibrary

true

If enabled, Mirror121 draws charts using Google JS API. If disabled Mirror121 does not draw charts at all and does not communicate with that API at all.

7.6. Memory Settings

Mirror121 allows you to configure how much memory it can use. After the start, Mirror121 allocates 700MB of memory and is allowed to allocate up to 2GB of memory.

In some cases you may need to increase a memory allocated to Mirror121: If you download a lot of big records at once or if you increased the number of synchronizations running in parallel (mirror121.quartz.threadCount property).

7.6.1. Windows service

  1. Open cmd.exe from Windows Start menu as an Administrator

  2. Type: cd "C:/Program Files/Mirror121/bin"
    (C:/Program Files/Mirror121 is default installation folder which is configurable during the installation)

  3. Open configuration window by typing: tomcat9w.exe //ES//Mirror121SF
    (Mirror121SF is a name of the service - if you changed the name during the installation, change it in this command)

    increase memory
  4. Open "Java" tab and increase the value in "Maximum memory pool" field. Click OK.

  5. Restart the service.

7.6.2. Linux

  1. Stop Mirror121

  2. Edit file /opt/Mirror121/mirror121.properties

  3. Add or edit the following line. Specify the value in megabytes (e.g. 1500m) or gigabytes (e.g. 2G):

    mirror121.memory.Xmx=2G
  4. Save the file

  5. Start Mirror121

8. How To

This chapter contains a bunch of mutually unrelated "How To" guides which don’t fit into any other chapter.

8.1. Thread Dump

Thread dump captures the current state of Mirror121 and its threads. It allows us to understand what was going on inside Mirror121 at moment you captured it. That is useful in case Mirror121 gets stuck. You are not expected to understand thread dump output - that is job for our L3 support.

The next two chapters describe how to create a thread dump on Windows and Linux.

How To Create Thread Dump On Windows

Version: 4.3.0 and newer, OS: Windows

  1. Open Task Manager (Ctrl + Shift + Esc)

  2. Click "Details" tab

  3. Find "tomcat9.exe" and remember the value of "PID" column

  4. Open cmd.exe as an Administrator

  5. Execute: cd "C:\Program Files\Mirror121"

  6. Execute: jre\bin\jstack.exe PID > logs\thread-dump.log

  7. Send us C:\Program Files\Mirror121\logs\thread-dump.log file

thread dump windows

How To Create Thread Dump On Linux

Version: 4.3.0 and newer, OS: Linux

  1. Connect to your server.

  2. Find out PID of Mirror121: ps aux | grep [/]opt/Mirror121 | awk '{print $2}'

  3. Create a thread dump file: /opt/Mirror121/jre/bin/jstack PID > /opt/Mirror121/logs/threaddump.log

  4. Send us /opt/Mirror121/logs/thread-dump.log

thread dump linux

9. FAQ

  1. What is the default timezone?

    Application uses UTC as default timezone. Also, the data stored in the database are in UTC timezone.

  2. Can I delete old log files about synchronization runs?

    Yes, you can. They are located by mirror121.activityLog.dir property (default value is logs/activityLogs). The files are located by this pattern: <synchronizationId>/<yyyy-MM>/<logId>.log. If you find the activityLogs directory too large you can delete files or whole directories to save some disk space.

  3. How do you protect my passwords?

    Every sensitive information such as password is encrypted with AES-256 cipher. Encryption is applied on data stored in database as well as data stored in configuration files.

10. Release Notes

Version Date Remarks

3.0.0

January 25, 2021

  • Mirror121 - the application has been renamed to Mirror121.

    • All files, directories and configuration options that contained the old name are automatically renamed during upgrade.

    • The only directory that we don’t rename is the "/opt/SnowMirror" directory.

  • Re-design - we have changed the appearance of the user interface so that it uses Mirror121’s visual identity.

2.5.2

January 15, 2021

  • Snowflake data types - we have changed a mapping of Salesforce data types to Snowflake data types. If you use Snowflake database, please review the changes. If you create a new column, it will use the new mapping. Existing columns are not affected.

2.5.1

January 11, 2021

  • Fixed: some synchronizations failed with "unexpected end of file while reading quoted column beginning on line 2 and ending on line 2" error.

2.5.0

November 13, 2020

  • Snowflake support - you can store your data into Snowflake warehouse.

  • Authentication using SSO - Mirror121 can authenticate users using Single Sing-On over SAML 2.0 protocol and Web SSO profile.

  • Improved performance: Mirror121 downloads data from Salesforce and stores it to database in parallel. Data is stored to the database in so-called "Data Processing" threads.

    This may improved the total time needed to finish a synchronization especially if the connection to the database is slow.

    serial vs parallel processing
  • Java Upgrade - Upgraded Java to the latest version 11.0.9+11.

  • Apache Tomcat - Upgraded the embedded web-server Apache Tomcat to the latest version 9.0.39.

2.4.4

June 15, 2020

  • Signed Windows exe installer with a new code signing certificate.

2.4.3

June 6, 2020

  • Fixed: Broken link in documentation.

2.4.2

June 3, 2020

  • Fixed: Names of Lightening attachments did not contain an extension. For example a file named "Report.pdf" was synchronized as "Report".

    • To load the extensions for already synchronized files you have to re-synchronize from scratch (i.e. call Clean And Synchronize)

2.4.1

May 19, 2020

  • Retry mechanism - Mirror121 downloads data into temporary file before storing them to the database. If there was a network error during the download the synchronization failed. We have added a retry mechanism. In case of error, Mirror121 tries to download the file up to 5times before failing.

  • Fixed: Checkbox "Change Proxy Password" on Settings → Salesforce page was show even when it should not have been show.

  • Fixed: In some cases, Mirror121 failed to properly check if table exists. Only tables containing "_" were affected.

  • Fixed: Count Records button on Edit Setting page may have shown incorrect result.

    • Only users using the following General Settings were affected: Records Delete = Soft and Synchronize Archived = No and Synchronize Deleted = Yes

2.4.0

May 13, 2020

How to upgrade

  • We have upgraded the embedded web-server Apache Tomcat. Because of that, the name of the process has changed from "tomcat8" to "tomcat9". Some customers may have a rule in firewall (or some other security tool) which allows/prevents the process from running. If you have such a rule please change it so that the new name "tomcat9" is permitted.

  • Linux users must upgrade Java to the latest AdoptOpenJDK release as described below in the Change Log. On Windows, Java is updated automatically when Mirror121 is updated.

Changelog

  • Upgraded Java to 11.0.6+10 - Mirror121 is written in Java and needs Open JDK to run. Since Java 11, Oracle (the company that owns and develops Java) has changed licensing terms and support model. This topic is quite complex but the important message is that you don’t have to pay for Java.

    Mirror121 for Windows already contains free of charge Java. Linux users are advised to pay attention to which Java distribution they download. There are several distributions of Java available:

    • Please, use Open JDK builds from AdoptOpenJDK.

    • Please, avoid Oracle JDK. OracleJDK distribution is paid and comes with Oracle’s commercial support.

  • Apache Tomcat - Upgraded the embedded web-server Apache Tomcat to the latest version 9.0.35.

2.3.1

October 24, 2019

- Fixed: Mirror121 had trouble reading trial licenses downloaded and generated by our website.

2.3.0

August 12, 2019

- Added a support for archived records. Users can choose how to treat archived records - synchronize them (default) or ignore them. Users should clean and synchronize tables that support archiving (Announcement, ContentDocument, Event, CollaborationGroup, Pricebook2, Task).

2.2.3

August 6, 2019

- Fixed: Consistency check on synchronizations with Truncate delete strategy incorrectly assumed that nothing could have been deleted.

2.2.2

August 2, 2019

- Fixed: Soft deleted records were not updated before marked as soft deleted.
- Fixed: Soft deletes are picked up together with inserts and updates. If Soft delete is used, delete strategy may be set to None.
- Fixed: Incremental load could have missed updated records when Diff sync had run before it.

2.2.1

July 29, 2019

- Fixed: Failure to load meta-data of a single table caused failure of all the other synchronizations.

2.2.0

July 23, 2019

- Fixed: Audit Deleted strategy could have missed some records deleted in Salesforce. Therefore, mirror database contained more records than Salesforce. Use Differential synchronization to remove excessive records.

2.1.10

July 17, 2019

- Tables without "SystemModstamp" and "LastModifiedDate" columns always triggered clean synchronization. From now on, if none of the columns is present the "CreatedDate" column is used instead.
- Allowed to use Audit Delete Strategy on history tables.
- Fixed: In some cases, picklist used to be mapped to data types with max. length of 80 characters. From now on, the max. length has changed to 255 characters.

2.1.9

July 8, 2019

- Fixed: Picklist used to be mapped to data types with max. length of 80 characters. From now on, the max. length has changed to 255 characters.

2.1.8

July 2, 2019

- Fixed: Data type "time" was not properly handled which lead to a synchronization failure.

2.1.7

June 27, 2019

- Before storing data to the database, Mirror121 stores data from Salesforce into temporary file. That minimizes the time the connection is open and prevents network issues.

2.1.6

June 21, 2019

- Using "SystemModstamp" column instead of "LastModifiedDate" to synchronize data because it covers changes from automated processes.

2.1.5

June 13, 2019

- Fixed: Incremental synchronization failed when an "Id" column was renamed to "ID"
- Fixed: "jsonNot a valid enumeration for type: class com.sforce.soap.partner.SoapType" error. Fixed by updating "com.force.api" libraries to the latest versions (which support API v46).

2.1.4

June 7, 2019

- Introduced an advanced property "mirror.mssql.useUnicodeDataTypes" which allows you to use "nvarchar" data type instead of "varchar" on SQL Server.
- Introduced an advanced property "mirror121.dataType.inheritPrecision" which allows you to use precision and scale of decimal type as it is defined in Salesforce.
- Introduced a new Auto Schema Update option - Enabled (no truncation). It allows you to add new columns without cleaning and synchronizing the whole table.
- Fixed: Synchronization failed due to error: null

2.1.3

May 29, 2019

- Added an option to use uppercase mirror table names. It can be configure on Settings → Advanced Settings page by setting "mirror121.mirrorTable.namingStrategy" property.

2.1.2

March 27, 2019

- Soft Delete - besides "mirror_deleted_on", it is possible to add another column "mirror_deleted". It contains a flag which helps you to determine if a record was deleted or not. This is configurable through Advanced Properties.

2.1.1

March 27, 2019

- Fixed: Differential sync fail because it could not find mirror_deleted_on column.

2.1.0

March 27, 2019

- A new option in General Settings - ability to automatically remove suffixes (e.g. "__c") from column names in mirror database.
- Soft Delete - instead of deleting records, you can optionally mark them as deleted and keep them in mirror database.

2.0.2

February 26, 2019

- Security: Added ability to enforce strong password.
- Security: Changed pages where passwords are entered so that they adhere best practices.

2.0.0

August 15, 2017

- Upgraded and improved replication algorithm so that it can handle bigger tables.
- Besides replication we also support backups.

1.1.0

February 20, 2015

- Added password encryption - credentials to config database, mirror database and Salesforce are encrypted using AES 256.
- Improved UI - all necessary information is visible right away (e.g. status bars).
- Automatic detection of user’s timezone when creating a new user.
- Minor bug fixes for SQL Server and PostgreSQL databases.

1.0.0

January 27, 2015

Public release.