Upgrade MicroStrategy OCF Connector to Version 1.2.0¶

Alation Cloud Service Applies to Alation Cloud Service instances of Alation

Customer Managed Applies to customer-managed instances of Alation

After upgrading the MicroStrategy OCF Connector to version 1.2.0 or higher from an earlier version, you must execute the migration script once. If you are on Alation version 2024.3.4 or earlier, you must manually run the migration script. Migration ensures transfer of all curated information on the catalog pages of MicroStrategy objects from your previous connector version is accurately transferred to the MicroStrategy OCF Connector version 1.2.0.

Note

If you are installing the MicroStrategy connector for the first time and choose to use the connector version 1.2.0 or higher, you needn’t perform this migration. The script is only required if you are upgrading the connector from a previous version.

For Alation Version 2024.3.4 and Higher¶

Upgrade to Connector Version 1.2.0 or higher and run MDE.

Migrate native to OCF connector. Alation executes the curation migration script automatically as part of the first MDE run after migrating to OCF connector version 1.2.0 or higher. This curation migration runs as a downstream job as part of the first MDE after migration. Verify the migration results in the Job_details section, see Migration Results section for more details.

For Alation Version Prior to 2024.3.4¶

Upgrade: Upgrade to Connector Version 1.2.0 or higher.

Run Metadata Extraction.

Ensure that MDE runs successfully. Once completed, make a note of the first_ocf_mde_date parameter in the format: 'dd-mm-yyyy'. For example: '25-12-2023'. This date indicates the first OCF MDE run after upgrading the connector and not when it was completed. This is important for accuracy in the curation data migration.

Run the Curation Script: After upgrading the connector, you must manually execute the curation migration script to restore any required missing curation data following the subsequent MDE run.

Refer to the connector logs for errors, if any. Troubleshoot the errors before running the curation migration script.

Note

In case of native to OCF migration to connector version 1.2.0, execute the curation migration script manually following the subsequent MDE run.

Run the Curation Migration Script¶

Note

This section only applies to the Alation versions prior to 2024.3.4 and OCF connector version 1.2.0 or higher.

Important

For Alation Cloud Service, you must raise a support ticket to run the curation migration script.
For on-premise installation, contact Alation Support to get access to the curation migration scripts.

Script Arguments¶

-s—BI Server datasource ID from the source URL.
- Example: if the BI source URL is https://aaa.alationcloud.com/bi/v2/server/13/, the ID is 13.
first_ocf_mde_date—Date on which OCF MDE was first triggered after upgrading the connector and not when it was completed
- The format must be 'dd-mm-yyyy' e.g. '16-11-2023'

Run the Script¶

To run the script:

Use SSH to connect to the Alation server.
Enter the Alation shell.
sudo /etc/init.d/alation shell
Navigate to the ingestion directory and place the curation migration script received from Alation Support.
cd /opt/alation/django/bi_metadata/ingestion/
# assuming that the script was uploaded into the /tmp folder on the Alation machine
cp /tmp/copy_curation_data_mstr.py .
Navigate to the one_off_scripts directory and place the curation migration script received from Alation Support.
cd /opt/alation/django/rosemeta/one_off_scripts/
# assuming that the script was uploaded into the /tmp folder on the Alation machine
cp /tmp/copy_mstr_logical_data_from_deleted_objects_to_new_objects.py .
Change the user to alation.
sudo su alation
Select one of the following modes:

Dry-run — Executes the script without actually making any database writes, only reads.

nohup python -m copy_mstr_logical_data_from_deleted_objects_to_new_objects -s <bi_server_id> -first_ocf_mde_date '<date>' &

Confirm — Makes permanent changes to the database and is not reversible. It should only be run once.

nohup python -m copy_mstr_logical_data_from_deleted_objects_to_new_objects -s <bi_server_id> -first_ocf_mde_date '<date>' --confirm &

Example: nohup python -m copy_mstr_logical_data_from_deleted_objects_to_new_objects -s 2 -first_ocf_mde_date '16-11-2023' --confirm &

Note

The script execution is a database-intensive as well as time-consuming operation based on the volume of data on the BI Server.

Use the command to run the script in the background and output a PID. Use this PID to check the status.

Track the progress of the script using its PID (process id).

ps -p 14496 -o %cpu,%mem,cmd

%CPU %MEM CMD
98.9  0.1 python -m copy_mstr_logical_data_from_deleted_objects_to_new_objects -s 3 -first_ocf_mde_date 25-10-2023 --confirm

Migration Results¶

The migration of curation data results will be displayed as part of MDE job details.

In case of any Curation errors, it will be displayed in the Job errors table as Migration error. Click Generate Error Report button to download the error report. In such cases, the MDE status will be Partial Success due to errors.

Users can perform the migration again in the following methods in case of an MDE failure:
Curation Migration API.
Run the Curation Migration Script - For Alation version 2024.3.4 and newer, copying the migration script and passing the first_ocf_mde_date argument is not required.
EXAMPLE:
nohup python -m copy_mstr_logical_data_from_deleted_objects_to_new_objects -s 2 --confirm &

The curation script migration logs are located in the directory /tmp/MicroStrategy(Old)_to_MicroStrategy(New)_migration.log. After the migration completes, review this log file for any URLs of the objects that could not be migrated. An object may not be migrated:

Most commonly because Alation could not find its match in the OCF source, for example when it was renamed or its parent was deleted.
If any URLs are printed, they will show the URL and curation that was missed:
http://localhost:8000//bi/v2/datasource_column/5/ | curation type: {Mark (star/watch)
If no URLs are found and/or the error keyword is not found in the logs, then you should see the message Completed the curation migration without any lost curation in the log file.

When you run the migration script after the native to ocf(connector version 1.2.0 or higher) migration or ocf to oct migration(connector version 1.2.0 or higher), it will list all objects that were not matched and their URLs. Use the URLs to view the objects and their logical metadata if you have to manually transfer the curation information for these objects to the corresponding pages of the objects under the now migrated OCF BI source

Understand the Curation Migration Report¶

Example

**** Searching for MSTR(old) to MSTR(new) matches for Data Sources

-----------------------------------------------------------------

Data Sources - Found a total of: 21 instances of curation

The following are the URLs (count: 1) to the Data Sources which we could not automatically match to a MSTR(new) object. They need to be reviewed (for validity) and to migrate their curation manually.

http://localhost:8080//bi/v2/datasource/195/ | curation type: {'Mark (star/watch)', 'ValueHistory (oid)', 'GenericFieldValue (anchor tags)', 'Tag'}

2024-12-10 11:13:25.454574 —— ========================== Data Sources Curation Migration End ==========================

Explanation

Line 2: Indicates we’re searching for matches between native or OCF(old) and OCF(new) BI objects

Line 5: Indicates that we found 21 instances of curation on datasource objects

Line 7: Indicates the number of BI objects (and how many BI objects) for which we couldn’t find a new OCF match and so cannot copy curation to it (OCF object)

Line 9: Indicates the URL for the object we could not migrate curation and the type of curation in brackets. Review the URLs and migrate the curation data on them manually.

Troubleshooting¶

For Connector settings migration errors, refer to the migration log file: /opt/alation/site/logs/ypireti.log
Permissions error: Ensure that you are running the script as the user alation
Django imports errors or issues: Ensure that you run the script from the directory /opt/alation/django/rosemeta/one_off_scripts/
For any other issues, contact Alation Support with the following log file attached to the Support ticket: /tmp/MicroStrategy(Old)_to_MicroStrategy(New)_migration-debug.log